Hallucination Analysis in Large Language Models for Arabic Question Answering

Recently, Natural Language Processing (NLP) has gained significant popularity and is continually evolving. As a crucial link between humans and computers, NLP enables machines to comprehend, translate, and generate human language [1]. This evolution has contributed to the emergence of Large Language Models (LLMs) like BARD (Gemini Now) and ChatGPT, which enhance conventional chatbot capabilities through Deep Learning (DL) and transformer architectures [2]. LLMs find applications across various domains, including search engines, translation, medical diagnostics [3], code generation [4], higher education [3, 5], finance [6], and sport [7]. However, despite their impressive performance, a key challenge known as "hallucinations" hinders their practical reliability and precision as sources of information. In the field of computing, hallucinations refer to instances in which LLMs produce responses that sound convincing but are actually incorrect or nonsensical [8, 9]. Simply put, these hallucinations happen when the model creates information that seems believable yet isn’t grounded in reality. This can include fabricating details, making up events, or straying beyond what it has learned from its training data, leading to inaccurate outputs. These responses may appear authentic but lack any basis in real-world facts [10]. Tackling this issue is essential for improving the reliability and trustworthiness of AI-generated content. Hallucination poses a significant challenge in medicine [3, 11], business decision-making [12], and other fields that require highly accurate measurements [13]. Looking back to the root causes of hallucination LLMs, researchers found that models deliver statistically based outcomes due to limited real-world knowledge, bias, or deceptive training data. This suggests a potential lack of understanding of input [14]. Furthermore, Hallucinations in LLMs affect their significant economic potential and rapid adoption. Recent research conducted by McKinsey in June 2023 highlights the economic growth and productivity gains enabled by AI generative applications. According to their estimates, this technology has the potential to annually generate value for the global economy ranging from \$2.6 trillion to $4.4 trillion [15]. The introduction of ChatGPT by OpenAI in late 2022 garnered widespread attention for chatbot technology among the public. Within just five days of its launch, it amassed one million users. Notably, a milestone was achieved in January 2023, with the platform attracting over 100 million users [16].

Moreover, the growing adoption of LLMs is reaching diverse communities, including Arabic speakers, representing a significant segment of internet users. Arabic is the fourth most spoken language in the world, with over 480 million native speakers [17, 18]. Approximately 67% of the Arab world’s population are internet users [19], increasing demand for language processing technology. Evaluating LLMs in the context of the Arabic language has both practical applications and significant commercial relevance. Hence, developing LLMs that can understand and generate Arabic is vital for inclusive AI advancements. However, challenges such as a lack of high-quality Arabic datasets and dialectal variations hinder these models’ effectiveness [20]. Consequently, hallucinations in AI outputs lead to inaccuracies and risk spreading misinformation among Arabic users, undermining the reliability of AI applications.

Current research works in the field of LLM hallucinations are examining the underlying factors and exploring methods to mitigate them. Detecting misinformation generated by LLMs is more challenging than detecting that generated by humans [21]. This highlights the importance of understanding the core of the problem and how different LLM models behave. Most LLM models are developed for Latin Languages, making it challenging to handle non-Latin environments [22]. Moreover, the size of training data plays a critical role in effective models where many non-Latin Languages have smaller sizes compared to Latin Languages [23-28]. To our knowledge, no previous academic studies have been conducted on the phenomenon of hallucinations of LLM in the Arabic language to date.

To address this gap, this research studies the phenomenon of hallucinations generated by LLMs in Arabic language, focusing on question-answering tasks with the primary aim of contributing to the continued advancement of the field. The main contributions of this study are as follows:

• To investigate hallucinations in Arabic language QA task in public prominent LLMs: Gemini, ChatGPT-4.0, ChatGPT-3.5, Claude 3.5, and Jais.
• To compare the tendency of hallucinations in the QA task in five prominent public LLMs for Arabic and English languages.
• To evaluate the hallucination rate of a black-box model adopting a sampling-based approach.
• To analyze the hallucination level in various generated responses.

The remaining sections of the paper are organized as follows: Section 2 presents the problem statement and research motivation. Section 3 provides background on NLP and Hallucination in LLMs. After that, Section 4 discusses recent studies on hallucination in LLMs across different tasks. Section 5 then introduces the research methodology. Following the presentation of the results and discussion in Section 6. Finally, Section 7 presents the conclusions and potential future work.

Natural Language Processing (NLP) is an AI field that processes speech and text to understand their syntactic, semantic, and emotional aspects [29]. Rule-based techniques were initially used [30, 31], followed by supervised and unsupervised approaches [32-34], and most recently, transformers bring a breakthrough in NLP tasks based on an encoder-decoder structure along with long-range dependency capture [35-37]. The development of LLMs has revolutionized NLP, achieving human-level performance on various language tasks, including medical diagnostics [3], code generation [4], higher education [3, 5], and Finance [6]. Despite the outstanding performance of these models, a fundamental issue called "hallucinations" has been preventing their practical usage as a reliable source of information. The subsequent sections outline Hallucination in LLMs, including its taxonomy, causes, and mitigation strategies.

3.1 Hallucination definition

In a computing context, the term hallucination is defined as " a plausible but false or misleading response generated by an artificial intelligence algorithm" [8]. The LLM Hallucinations occur when a model produces false or fabricated information, straying from factual knowledge and giving answers unsupported by the model’s training data or the input provided [9]. This phenomenon can arise due to various factors related to the data, training processes, or inference methodologies, as discussed in subsection 3.3.

3.2 Classification of hallucinations

The categorization of LLM hallucinations remains an evolving field. While no single taxonomy reigns supreme, researchers have proposed diverse categories to classify them. This section reviews the different classification approaches. Figure 1 offers a detailed overview of the various classifications of hallucinations based on reviewed studies. Also, examples for each category are presented in Figure 2.

Figure 1. Overview of hallucination classification

A study [30] has identified two forms of hallucinations in natural language generation. The first category is Intrinsic hallucination which happens when an LLM produced output contradicts the original content. In contrast, the second category, extrinsic hallucination, arises when LLM produces outcomes that cannot be validated against the original content and lack support or contradiction within the original context.

Another study [14] introduced different types of Hallucination by categorizing hallucinations into two main groups. The first is factuality hallucination, which highlights the disparity between the outcomes produced and verified real-world facts. This category is divided into two subcategories based on verifiable sources. The first subcategory is factual inconsistency, which occurs when the AI provides outputs that include facts that can be confirmed but don’t quite add up. The second subcategory is factual fabrication, which occurs when the AI presents information that cannot be verified and contradicts what is known to be true. This type can sometimes appear as either unverifiability or an overclaim of facts.

Figure 2. Examples of each category of large language model (LLM) hallucinations along with an explanation

Note: Text highlighted in Red represents the hallucinatory generated output, while text highlighted in Blue represents user prompts or provided context that contradicts the hallucinations generated by the language model [14].

The other category is faithfulness hallucination, which deals with how well the AI’s generated content aligns with what the user has asked for. Users often experience inconsistencies in three ways: instruction, context, and logical inconsistencies. Instructional inconsistency occurs when the AI’s responses diverge from what the user intended, highlighting instances in which the AI misinterprets benign instructions, even if some of those deviations are related to safety measures. Context inconsistency refers to the AI contradicting the context provided by the user. Finally, logical inconsistency can arise when AI output contains contradictions, which is particularly noticeable during tasks that require reasoning.

3.3 Hallucination causes

Hallucinations in LLMs stem from multiple factors associated with their comprehensive capability development. The primary origins of these hallucinations are data, training, and inference. Pre-training data is crucial, as it serves as the basis for these models, enabling them to develop general skills and factual knowledge [36]. However, pre-training data can inadvertently contribute to hallucinations in LLMs through two primary mechanisms: risks associated with imperfect or unreliable data sources and the inadequate application of factual knowledge contained within the data. LLMs undergo two key phases of training: the initial pre-training phase, during which they develop general representations and acquire broad world knowledge, and a subsequent alignment phase aimed at refining their responses to align better with user instructions and preferences. Shortcomings in either of these phases can result in hallucinations in the model’s outputs [14]. Decoding plays a vital role in demonstrating the capabilities of LLMs after pre-training and alignment. Therefore, faulty decoding representation and the inherent randomness of decoding strategies can lead to hallucinations in LLMs [14].

3.4 Hallucination mitigation

The widespread use of LLMs presents a challenge when it comes to mitigating hallucinations. Studies have proposed various mitigation methods, but some can worsen hallucinations [37, 38]. A study [37] introduced a practical approach to reducing GPT3.5 hallucinations by rectifying false information in generated sentences using retrieved knowledge. Different categories of mitigation methods were suggested, classified into the following groups:

• Fine-tuning: A Machine Learning (ML) technique is employed to fine-tune a pre-trained model for a specific context using relatively small training data [39]. A study [40] demonstrated the effectiveness of fine-tuning in mitigating LLM hallucinations, although the computational cost can be prohibitive due to LLMs’ large parameter count.
• Knowledge Graphs: These methods enable the integration of structured and unstructured knowledge, providing LLMs with a broader platform to perform tasks [41]. However, designing and maintaining a curated knowledge base can require considerable time and effort.
• Memory Augmentation: Given the increasing need for deep learning techniques to expand their functionalities by incorporating additional knowledge, a recent study [42] developed an enhanced transformer specifically designed for knowledge-intensive NLP tasks. Despite the proven benefits of memory augmentation in NLP models, no similar tests have been conducted for LLMs.
• Context Prompts: A self-monitoring prompting framework was introduced that leverages formal methods for error identification and response alignment [43]. Another study [44] introduced Self-Familiarity, a zero-resource pre-detection method designed to reduce the risk of LLMs generating inaccurate output.
• Pre-emptive Strategies: The study by Feldman et al. [45] developed a method utilizing context-tagged prompts to improve LLM response accuracy by guiding them with context prompts and validated questions.

Each approach offers unique potential for mitigating LLM hallucinations, contributing to ongoing efforts to enhance the reliability and trustworthiness of LLM-based systems.

The subsequent sections present the research methodology for investigating Hallucination in Arabic LLMs for question-answering (QA) tasks with two experiments. The first one evaluated five recent and prominent LLMs, ChatGPT-4.0 and ChatGPT-3.5 [74], Gemini [75], Claude 3.5 [76], and Jais [77] using Hallucination triggering questions from the HaluEval benchmark [78]. To evaluate these models’ tendency to produce hallucinations, an online prompt is utilized to pose the question in both Arabic and English. Additionally, precise criteria are specified to determine whether the response displays hallucinations. The overview of the first experiment is presented in Figure 4.

Figure 4. First experiment’s framework for evaluating hallucination in Arabic

For the second experiment, an Arabic open-domain question answering dataset is used to assess GPT-4o-mini-2024-07-18 [79]. The model selection is due to the version being a lighter variant of GPT-4o that performs efficiently while maintaining low resource consumption. Then, the selfcheckGPT framework is utilized to detect Hallucinated answers. Lastly, evaluation is performed using the BertScore metric employing the bert-base-multilingual-cased model variant to measure the similarity between the correct and generated answers. The suggested framework is shown in Figure 5. The subsequent subsection presents in detail the implementation of these two experiments.

Figure 5. Second experiment’s proposed framework for evaluating hallucination in Arabic

5.1 First experiment

5.1.1 Dataset

A set of 12 questions is meticulously selected from the HaluEval benchmark dataset [78], a comprehensive collection containing 5,000 common user questions and 30,000 domain-specific samples covering areas like QA, dialogue, and summarization that induced Hallucination in LLMs. Experiments conducted with the HaluEval benchmark shows that not all prompts lead to hallucinated responses. A study tested and filtered the HaluEval dataset, carefully selecting 20 questions that triggered hallucinations in one or more of the chosen LLM [25]. These 20 questions were categorized into three classes: Chinese context-dependent (5 questions), English context-dependent (3 questions), and language-independent contextual (12 questions). Based on these classifications, this study extracted 12 questions from a total of 20, focusing exclusively on those categorized as language context-independent. Then, following the pattern established by the previous study to explore specific language contexts, two questions were generated that are Arabic context-dependent. Thus, questions 1 through 12 are language context-independent, while questions 13 and 14 are Arabic context-dependent. A total of 14 questions are presented in Figure 6. These questions refer to general knowledge and span multiple fields, including common sense, technology, mathematics, language analysis, geography, and science. This experiment explores the impact of language on LLMs’ hallucinations by examining both the input prompt’s language and the cultural context it implies. The experiment is conducted in English and Arabic, categorizing cultural background into Arabic context-dependent and language context-independent. Several factors inform this choice. First, the selected LLMs in this experiment can be categorized into Arabic LLMs, such as Jais, and multilingual LLMs. Since English is considered a high-resource language while Arabic is classified as low-resource [80, 81], this allows for a comparison of hallucination evaluation comparison based on corpus size [82]. Second, the way language is used and expressed in Arabic differs from that in English [83], which is vital to ensure the wide usability of the LLMs.

Figure 6. The sample of 14 questions used to induce hallucination in large language model (LLM) in the first experiments is shown in English and Arabic

5.1.2 Large Language Models selection

In the first experiment, five pre-trained LLMs were utilized for evaluation purposes. The LLMs used were ChatGPT-4.0 and ChatGPT-3.5 [74], developed by OpenAI; Gemini [75], developed by Google; Claude 3.5 [76], developed by Anthropic; and Jais [77], launched in 2023. Jais was specifically trained with 13 billion parameters to cater to the intricacies of the Arabic language, enhancing its understanding and generation capabilities in that linguistic context. All LLMs’ calls were tested using their standard online settings, without any adjustments for specific tasks. To keep things consistent, the experiment followed a standard mid-range setup across the different models. Hence, setting the temperature value to be 1.0, the top-p value to be 0.95, and the top-k value to be 40 when applicable. The maximum number of tokens generated was determined automatically by the platform.

5.1.3 Evaluation metrics

The evaluation process was carefully structured to assess the tendency to hallucinate across different LLMs in the Q&A task. Each model was provided with a set of 14 questions in Arabic. The same set of questions was also presented in English for comparison. The responses were analyzed to distinguish between accurate answers and those that exhibit Hallucination. Responses were categorized as either exhibiting hallucinations or not, based on defined criteria. Given the correct answer or possible answers in case of open questions, the criteria used to determine whether an answer Exhibits Hallucination or Does Not Exhibit Hallucination are as follows:

A. Exhibits Hallucination: A response is classified as exhibiting Hallucination if it meets any of the following conditions:

• Lack of Fluency: The response is incoherent, ungrammatical, or contains excessive nonsensical content.
• Irrelevant Correctness: While the response contains largely accurate information, it does not directly address the question posed.
• Inference Violation: The response includes statements that cannot be logically inferred from the correct answer examples or contains information that contradicts them.

B. DoesNotExhibitHallucination: A response is considered free from Hallucination if it satisfies either of the following conditions:

• Logical Consistency: The response is fully supported or can be reasonably inferred from at least one correct answer example.
• Adherence to Undefined Cases: If correct answer examples indicate that a question cannot be answered, then a generated response such as "I do not know" is classified as non-hallucinatory.

Since the criteria for Hallucination and non-hallucination are inherently contradictory, a response that meets the conditions for one category cannot logically belong to the other. We assess hallucination using a scoring system, where each response exhibiting Hallucination adds 1 point to the model’s hallucination hallucination score. This score ranges from 0 to 14, with higher values indicating a greater tendency to generate hallucinated content. To maintain fairness, we present each question 3 times, each with a new prompt, to prevent any influence from previous dialogues. This structured evaluation framework ensures a rigorous and objective assessment of response validity, prioritizing accuracy, relevance, and logical coherence [66].

5.2 Second experiment

5.2.1 Dataset

The open-domain question for this experiment is utilized from the publicly available wide scale ArabicaQA dataset for open domain QA and machine reading comprehension in Arabic [84]. The authors followed carefully crafted methodology steps to create high-quality question and answer pairs, which include article selection from wiki data, question generation, filtering, and human evaluation. The data set covered many aspects, including people, dates, organizations, products, and locations. We focus on open-domain tasks, which contain more than 80,000 pairs of questions and answers divided into training, development, and testing sets. Due to resource constraints, we conducted our evaluation on a subset of 400 questions from the ArabiaQA dataset. This selection aligns with previous research that utilizes LLMs for evaluation purposes of a given task [67, 68]. The subset was selected using random sampling with a fixed random seed.

5.2.2 Sampling based approach

One of the constraints of LLMs’ hallucination detection approach for low-resource languages is the scarcity of annotated data with an appropriate size and format for a specific task. Furthermore, a previously suggested method requires the capability to view token probability distributions, which could be unavailable for black box models. A sampling-based approach with a reference-free framework is suggested in the previous study [85].

5.2.3 Implementation details

We implement the SelfCheckGPT framework utilizing GPT-4o-mini [79]. Starting by randomly selecting 400 questions from the ArabicaQA dataset. Following that, we assess the model in a zero-evaluation manner by prompting it to generate a primary response with a temperature equal to 0 and three sample responses with a temperature equal to 1, both having a token limit of 100, resulting in a total of 1600 API calls. Bert-base-multilingual-cased variant [86] has been used to measure BertScore from the generated responses. For sentence splitting, we used the NLT sentence tokenization function instead of Spacy, which was used in the SelfCheckGPT repository to provide reliable sentence segmentation for Arabic languages.

This study explored hallucinations, where LLMs generate fabricated or inconsistent responses, focusing on the QA task in the Arabic language and presenting a thorough overview of the definition, classification, causes, and mitigation of LLM hallucinations. To examine the phenomenon of LLMs’ hallucination in question-answering tasks within the Arabic language, this study undertook two experiments. The first experiment was predicated on zero-shot learning, employing LLM models such as ChatGPT-3.5 and Gemini. The second experiment implemented a sampling-based approach by utilizing the selfcheckGPT framework. The findings of the first experiment shed light on the tendency of five prominent LLMs to produce hallucinated responses in Arabic and English. The second experiment applied the BERT Score to measure the similarity between the model’s responses, and lower scores indicate hallucinated responses. Overall, this study provides insight by investigating the level of Hallucination of LLMs for the Arabic language, which poses unique challenges compared to other languages, such as morphological richness, script variations, and lack of annotated datasets [70, 71].

These challenges are characteristic of many low-resource languages, where limited high-quality training data, scarce benchmarks, and diverse dialectal variations hinder the development of reliable language models. Morphologically complex languages, including Arabic, often require models to understand intricate root-and-pattern systems, extensive inflectional paradigms, and context-dependent word forms, which can increase the likelihood of hallucinated outputs. Additionally, the scarcity of annotated corpora for both formal and dialectal varieties limits opportunities for model fine-tuning and evaluation, further exacerbating performance gaps compared to high-resource languages like English. As observed in studies of other low-resource languages, such as Chinese [66] and Bulgarian [24], hallucination rates tend to be higher, particularly in domains that require factual precision or context-sensitive reasoning. By focusing on Arabic, this research highlights the broader need for improved resources, evaluation benchmarks, and targeted strategies for low-resource languages, contributing to more accurate and reliable NLP systems across linguistically diverse contexts.

A key limitation lies in the experiment’s scale. Testing only a few LLMs on a small dataset limits the generalizability of the findings. Different languages possess unique grammatical structures, expressions, and cultural contexts that can influence hallucinations in varying ways. Future work should involve testing a wider range of LLMs across diverse languages and larger datasets to gain a more comprehensive understanding. This would allow researchers to explore the influence of language on hallucinations, delve deeper into the root causes, and develop methods for improving LLM robustness.