## DECEPTIVE AI SYSTEMS THAT GIVE EXPLANATIONS ARE MORE CONVINCING THAN HONEST AI SYSTEMS AND CAN AMPLIFY BELIEF IN MISINFORMATION

## Valdemar Danry

## Pat Pataranutaporn

MIT Media Lab Massachusetts Institute of Technology Cambridge, MA 021239 vdanry@mit.edu , USA

## Matthew Groh

MIT Media Lab Massachusetts Institute of Technology Cambridge, MA 021239, USA patpat@mit.edu

## Ziv Epstein

Kellogg School of Management Northwestern University Evanston, IL 60208, USA

Institute for Human-Centered AI Stanford University Stanford, CA 94305, USA

Pattie Maes MIT Media Lab Massachusetts Institute of Technology Cambridge, MA 021239, USA

August 2, 2024

## ABSTRACT

Advanced Artificial Intelligence (AI) systems, specifically large language models (LLMs), have the capability to generate not just misinformation, but also deceptive explanations that can justify and propagate false information and erode trust in the truth. We examined the impact of deceptive AI generated explanations on individuals' beliefs in a pre-registered online experiment with 23,840 observations from 1,192 participants. We found that in addition to being more persuasive than accurate and honest explanations, AI-generated deceptive explanations can significantly amplify belief in false news headlines and undermine true ones as compared to AI systems that simply classify the headline incorrectly as being true/false. Moreover, our results show that personal factors such as cognitive reflection and trust in AI do not necessarily protect individuals from these effects caused by deceptive AI generated explanations. Instead, our results show that the logical validity of AI generated deceptive explanations, that is whether the explanation has a causal effect on the truthfulness of the AI's classification, plays a critical role in countering their persuasiveness - with logically invalid explanations being deemed less credible. This underscores the importance of teaching logical reasoning and critical thinking skills to identify logically invalid arguments, fostering greater resilience against advanced AI-driven misinformation.

## 1 Main

Artificial Intelligence (AI) systems, such as large language models (LLMs), have the alarming capability to generate not only misinformation but also deceptive explanations that justify misinformation and make it seem logically sound.

Researchers have identified an increase in AI-generated disinformation campaigns [1, 2] and the factors that make them disruptive to people' ability to discern true and false information [3, 4, 5, 6, 7, 8, 9, 10], as well as change people's attitudes [11, 12, 13]. These factors include authoritative tone [14], persuasive language [12, 14, 3], and targeted personalization [15]. However, little is known about the influence on people's beliefs when explanations are used as a tactic for misinformation.

In the broader context of the AI-generated misinformation landscape, we can identify three levels of sophistication, each posing unique challenges. The most basic level involves AI systems generating false news headlines without

Figure 1: Different levels of AI-generated misinformation: (1) AI-generated false news headlines, (2) AI-generated Deceptive Classifications, and (3) AI-generated deceptive explanations.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00024v1-figure-03f1c43b-410f-4e9d-84b7-8a4a0cfa9f31.jpg)

any accompanying justification. While such headlines can still mislead, they are more easily dismissed as baseless claims. The next level involves AI systems providing deceptive classifications, labeling false information as true or vice versa. This added layer of perceived authority can lend unwarranted credibility to the misinformation. However, the perhaps most subtle and deeper level is when AI systems generate deceptive explanations to justify and propagate false information.

One of the reasons why the topic of AI generated explanations and misinformation remains unexplored is that the use of explanations as a tactic for misinformation goes against the commonly held beliefs that explanations always make AI systems more transparent, trustworthy [16, 17], and fair [18, 19]. While researchers have shown that honest explanations can assist people in determining the veracity of information [20, 21] and improve their decision-making outcomes [22], as well as reduce human overreliance on AI systems [23], research in psychology has demonstrated that even poor explanations can significantly impact people's actions and beliefs [24, 25, 26]. This implies that the mere presence of an explanation can lead to changes in beliefs and behavior, regardless of its quality or veracity. Researchers have also shown that people often do not cognitively engage with the content of the explanation unless they are forced to do so[27].

Deceptive explanations can be used to exploit this vulnerability by connecting correct facts in misleading ways to justify false information or discredit real information and potentially making it more difficult to discern the truth. Disguised as political commentary, scientific explanations, or online discussions, AI-generated deceptive explanations from systems like LLMs could be weaponized at scale by bad actors to manipulate public opinion and decision making, even with safety measures built into the models [28].

The low cost and easy scalability of AI-generated explanations further compound the problem. With the ability to generate vast amounts of high quality content quickly and cheaply, malicious actors can flood online platforms with deceptive explanations, discrediting or drowning out legitimate explanations by human experts that take time to craft, making it harder for individuals to make sense of reliable information.

This paper investigates the effects of AI-generated deceptive explanations on human beliefs. Through a comprehensive pre-registered online experiment with 23,840 observations from 1,192 participants, we find that not only are deceptive AI generated explanations more persuasive than deceptive AI classifications without explanations, they are even more persuasive than honest explanations (See Fig. 2). While personal factors such as cognitive reflection and trust in AI do not necessarily protect individuals from the effects of deceptive AI generated explanations, the logical validity, that is whether the truthfulness of the conclusions or classifications made by the AI system follows from the truth of the explanation, of AI generated deceptive explanations where found to counter their persuasiveness - with logically invalid explanations being deemed less credible. This research underscores the urgent need for vigilance and proactive measures to ensure the responsible use of AI technology.

Figure 2: Top: Examples of how an AI system that helps users assess information can give an honest or deceptive explanation. Bottom: Procedure for assignment of stimuli domain (trivia items/news headlines, between-subjects), feedback type (AI-generated explanation/classification, between-subjects), and deceptive/honest, within-subjects).

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00024v1-figure-aaecf1cf-0321-4616-aebc-ae62ba26d408.jpg)

## Methods

## Stimuli Curation

We created a dataset of headlines each with one honest and one deceptive explanation by prompting the text-generation model GPT-3 davinci 2 with 12 example explanations randomly sampled from the publicly available fact-checking dataset 'liar-plus' [20]. This dataset consists of 12,836 short statements with explanation sentences extracted automatically from the full-text verdict reports written by journalists in Politifact (see Fig. 3).

First, 5 honest and 5 deceptive explanations were generated for 40 true and false headlines by prompting GPT-3 (davinci, temp = .7 ) with the headline and making it complete the sentences 'This is FALSE because. . . ' or 'This is TRUE because...' (see Fig. 4). We further curated the explanations by ranking them by highest semantic similarity and lowest repeated-word frequency. We picked the highest ranked explanations, confirmed the veracity and logical validity of each explanation. The truth veracity was confirmed independently by each of the authors after being instructed by a professional fact-checker following standard fact-checking procedures [29, 30] and then aligned. The logical validity was also confirmed by each author independently by deconstructing the claims of each headline and explanation into premises, conclusions and inferences from which the logical validity could be determined in an almost mathematical fashion using a Fisher analysis [31]. Next, we then excluded explanations whose veracity did not match the veracity of the headline. Since the resulting dataset had an unequal distribution of veracity and logical validity, we randomly excluded generated explanations until we had somewhat equal distribution of true and false explanations and logically valid and logically invalid explanations for each condition (deceptive vs. honest explanations) ending with a stimulus set consisting of 28 headlines with 1 honest explanation and 1 deceptive explanation each (56 total). We tested for differences across four linguistic dimensions (word count, sentiment, grade level, and subjectivity) and found no statistical differences between conditions. To generate explanations for a Trivia stimulus set, we repeated the same procedure by prompting GPT-3 with 12 example explanations. The example explanations and Trivia statements were randomly sampled from online[32], resulting in a stimulus set consisting of 28 trivia statements with 1 honest explanation and 1 deceptive explanation each (56 total). These were also tested for differences across four linguistic dimensions (wordcount, sentiment, grade level, and subjectivity) and were also found to have no statistical differences between conditions.

Figure 3: Examples of generated honest and deceptive classifications and classifications+explanations for whether a news headline or trivia statement is true or false.

|               |                                                                                   | Honest Explanation                                                                                                       | Decetive Explanation                                                                                                                                                                                         |
|---------------|-----------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| News Headline | "Planned Parenthood announced 'Abort-a-Palooza' on April 2" (Ground truth: False) | "This is FALSE. Planned Parenthood hasn't said anything about an 'Abort-a-palooza," according to the organization"       | "This is TRUE. Planned Parenthood announced it would be launching a nationwide bus tour to publicize its 'Abortion Without Apology' campaign, which will include stops in several cities, including Raleigh" |
| Trivia Item   | "The black box in a plane is black" (Ground truth: True)"                         | "This is TRUE. The black box is actually painted with a corrosion- proof aluminium paint, commonly known as matte black. | "This is FALSE. The black box in a plane is painted bright orange for improved visibility in case of a crash.                                                                                                |

In real-life scenarios, large language models may generate explanations without the filtering specified above. However, in this experiment, the filtering algorithm serves to ensure the quality and validity of the explanations for the purpose of our experiment. The aim was to isolate the effects of deceptive explanations and see how they manipulate beliefs even when at their best. Hence, it was important to minimize other variables such as irrelevance or incoherence of the explanations to get a normalized sample. Nonetheless, since the filtering algorithm is automatic, it has no selection bias and could potentially reflect how LLMs could be used in a controlled manner, either by malicious actors crafting algorithms to filter the best explanations to use in misinformation campaigns or by organizations using AI for generating honest explanations for automated fact-checking. In either contexts, it is very likely for there to be a layer of curation or filtering applied to create more convincing or high-quality content.

## Participant Recruitment

We recruited 1,199 participants through Prolific, https://prolific.co . Participants were required to self-report as US citizens, fluent in English and rated their fluency in any other languages they spoke. 1192 of these individuals passed an initial attention check task and were allowed to proceed. Additionally, all participants were fluent in English and 142 had fluency in a second language. Our final sample had a mean age of 39%, was 50% female, and was 72% white.

Figure 4: Examples of prompt engineering GPT-3 to generate honest and deceptive explanations for whether a news headline or trivia statement is true or false.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00024v1-figure-be17763f-9011-4144-9c71-fa86fc2f4955.jpg)

## Task Description

Participants were shown 20 statements during the main discernment task which were either true or false. Each participant saw the 20 statements in a random order, and rated the perceived truth of each statement ('Do you think the statement in the grey box is true or false?') on a slider scale with 1 decimal from 1 ('Definitely False') to 5 ('Definitely True'). After the rating, the participants would receive feedback from an AI system and be asked if they want to revise their rating ('Would you like to revise your estimate: Do you think the statement in the grey box is true or false?') on a slider scale with 1 decimal from 1 ('Definitely False') to 5 ('Definitely True') with the default value being same as the previous rating. Participants also rated their knowledge on the topic ('How knowledgeable are you on the topic of [topic]') on a slider scale with 1 decimal from 1 ('Not at all knowledgeable') to 5 ('Very much knowledgeable'). The selection of statements and generation of AI feedback is further explained in section 1 and the task interface can be seen in Fig. 5.

## Randomization

For the main discernment task, participants were randomly assigned to one of two conditions: (i) news headline statements or (ii) trivia item statements (between-subjects); and one of two conditions (i) no explanation ('This is true / false'), or (ii) explanation ('This true / false because. . . ') (between-subjects). The order of the stimuli being presented was also randomized. See Fig. 1 for examples of items across conditions.

## Post Task Survey

After the discernment task, participants were asked to complete post-test surveys to measure their critical thinking, and level of self-reported trust in the agent providing them with explanations. To measure the level of critical thinking of subjects, we used cognitive reflection test (CRT), a task designed to measure a person's ability to reflect on a question and resist reporting the first response that comes to mind [33]. For the CRT we randomly sampled three items from the extended CRT [34]. Finally, following Epstein et al. [35], to assess trust in the AI agent participants answered a battery of six trust questions derived from Mayer, Davis, and Schoorman [36]'s three factors of trustworthiness: Ability, Benevolence and Integrity (ABI).

## Participants

A total of 1,199 individuals participated in the experiment. We used the Prolific platform to recruit individuals from the United States. We focus our analysis on the 1,192 of 1,209 recruited participants who passed the attention check. 589 participants rating news headlines and 610 participants rating trivia statements. Of the 589 participants rating news headlines, 289 received no explanation and 300 received an explanation. Of the 610 participants rating trivia statements,

## Planned Parenthood announced 'Abort-a-Palooza on April 2

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00024v1-figure-2cdd8051-beb5-49f6-9697-4a9221c52bff.jpg)

## Al Explanation

TRUE Planned Parenthood announced it would be launching a nationwide bus tour to publicize its Abortion Without Apology' campaign; which will include stops in several cities; including Raleigh

Would you like to revise your estimate: Do you think the statement in the grey box is true or false?

Definitely False

Probably False

Probably True

Definitely True

2

3

4

5

Your choice

Figure 5: The impact of LLM based explanations ('This is true/false because...') compared to direct statements without explanations ('This is true/false') on participants' belief updates. Top Left: Honest explanations for news headlines. Top Right: Deceptive explanations for news headlines. Bottom Left: Honest explanations for trivia items. Bottom right: Deceptive explanations for trivia items.

299 got no explanation, and 311 got explanations. Each participant rated 20 statements, with an average of 51% of statements being true and 50% of explanations being deceptive.

## Approvals

This research complies with all relevant ethical regulations and the Massachusetts Institute of Technology's Committee on the Use of Humans as Experimental Subjects determined this study to fall under Exempt Category 3 - Benign Behavioral Intervention. This study's exemption identification number is E-3754. All participants are informed that 'This is an MIT research project. All data for research is collected anonymously for research purposes. We will ask you about your attitudes towards information and AI systems. For questions, please contact vdanry@mit.edu. If you are under 18 years old, you need consent from your parents to continue.' Participants recruited from Prolific were compensated at a rate of $10.82 an hour. At the end of the experiment participants were made aware that they had received AI explanations that were sometimes deceptive in the experiment, being told that "In this study, you were asked to collaborate with an AI-system for rating the accuracy of statements. All feedback in this study was AI-generated. Some of the feedback from the AI system was simply deceptive". Future work must explore the limits, ethics and consequences of exposing participants to AI-generated content.

## Scoring Perceived Truthfulness and Perceived Logical Validity

To investigate the effects of semantic features of explanations, we used GPT-4 to split each explanation into individual propositions / premises and score the perceived truthfulness and logical validity of each premise. For instance, for the headline 'Newsmax plans expansion to capitalize on Trump support, anger at Fox News', the explanation 'This is TRUE. Newsmax has been a vocal supporter of Trump, and the network has even hired Trump's former campaign manager.' was split by GPT-4 into the two premises: 'Newsmax has been a vocal supporter of donald trump.' and 'the network newsmax has hired donald trump's former campaign manager.'. To conduct the splitting GPT-4 was given the following prompt: 'Split the following explanation into its containing claims. Fill in pronouns and references so that each claim can be verified by itself without any context. Separate them with a new line. Only give the answer and be as concise as possible. Example: example. Explanation:explanation'.

Each premise of each explanation was then scored by GPT-4 for its perceived truthfulness using the following prompt: 'How likely is it that someone perceives this as true on a scale from 0.00-1.00 with decimals. 0.00 being extremely unlikely and 1.00 being extremely likely. Only give me the answer. Even if this is highly subjective how do you think some person might think this to be likely: premise'. Next, each premise was then scored by GPT-4 for its perceived logical validity with the deceptive classification of the headline using the following prompt: 'How likely is it that the general public would believe this: 'premise' to support this: 'This is classification: headline'. Output a score from 0.00-1.00 with decimals. 0.00 being extremely unlikely and 1.00 being extremely likely. Only give me the answer:'. An average score for both truthfulness and logical validity was calculated for each explanation.

## Calculating Syntactic Features

To calculate word count and reading ease, we used the Natural Language Toolkit (NLTK) in Python. The word count was simply the total number of words in each explanation. Reading ease was estimated using the Flesch Reading Ease formula, which assesses text on a 100-point scale; the higher the score, the easier the text is to understand.

To calculate the grammatical correctiveness of the explanation, we used the LanguageTool Python library, which returned the count of grammatical errors it could detect in the text.

## Analysis

In order to gain insights into whether explanations lead to more accurate beliefs, we first compare the belief ratings of user with AI-generated explanations (honest and deceptive) and no feedback. Accuracy was coded by subtracting the belief ratings from the ground truth per rating per participants.

To investigate the relationship between statement ground truth, the presence of explanations, and deceptive classifications (X1, X2, and X3) and belief and accuracy distribution (Y1, and Y2), along with their potential interactions and moderator variables, we employed an ordinary least squares (OLS) regression models.

We analyzed a total of 23,980 observations of belief ratings (12,200 trivia statement observations and 11,780 news headline observations) of true and false statements, collected through our experiment. The response variables, Y1 and Y2, represent the participant belief and accuracy distributions, while the predictor variables are as follows: X1 statement ground truth, X2 - presence of AI explanation, and X3 - deceptive AI feedback. The moderator variables

include the following z-scored variables: logical validity, self-reported prior knowledge, cognitive reflection test score [33], and trust in AI systems[35, 36].

We chose an Ordinary Least Squares (OLS) regression model for its analytical rigor in assessing linear relationships and controlling for confounding factors and mediators, making it ideal to intricately dissect the effects of AI explanations on belief accuracy and to unravel the complex dynamics between statement truth, explanation presence, and deception effectively. We assume that errors are independent and normally distributed with constant variance. Interaction terms between the predictors (X1:X2, X1:X3, and X2:X3) were included in the model to examine any joint effects of the veracity and explainability factors on belief distribution (Y1), accuracy distribution (Y2), and discernment distribution. Belief refers to the participants' subjective judgments about the truthfulness of the statements, while accuracy represents the correctness of these judgments (i.e., whether participants correctly identified true and false statements). We pre-registered our analysis at https://aspredicted.org/YLK\_S3F .

Our moderation analysis for CRT, need for cognition, trust, and prior knowledge was conducting by, for each moderating variable, re-running the main analysis model with the addition of the z-scored moderator and all interactions. Additionally, our moderation analysis also examined the 4-way interaction between veracity, explanation veracity, explanation type and the moderator.

We also conducting a moderation analysis for logical validity by re-running the main analysis model restricting to the classification + explanation condition, with the addition of the z-scored moderator and all interactions (limiting to only AI classifications with explanation) and examining the 3-way interactions between veracity, explanation veracity, explanation type and the moderator.

We then conducted a moderation analysis for number of premises, perceived truthfulness and perceived logical validity running the main analysis model restricted to deceptive explanations examining the interactions between headline veracity, number of premises, average perceived truthfulness of the premises and average perceived logical validity of the premises.

Lastly, we conducted a moderation analysis for the word count and reading ease by running the main analysis model restricted to deceptive explanations and examining the interactions between headline veracity, word count, and reading ease of the explanations.

## Results

In order to study the effects of deceptive AI explanations on human beliefs, we conducted a pre-registered online experiment with 23,840 observations from 1,192 participants rating their beliefs in true and false news headlines before and after receiving AI generated explanations. We designed our experiment to disentangle the influence of deceptive AI explanations from merely receiving a deceptive (inaccurate) classification of a news headline as true or false without explanation. This was done by randomizing participants to receiving true and false news headlines accompanied by either deceptive classifications or deceptive classifications with deceptive AI explanations (between-subjects). Moreover, we examine the impact of logical validity of the explanation, and how personal factors and the effects of syntactic and semantic features of the deceptive explanations may influence the outcomes.

For each true and false news headline, an LLM model (GPT-3) was used to generate an honest and a deceptive explanation by prompting the model to complete the following statements: 'This is false because. . . ' or 'This is true because. . . '. For deceptive explanations, the model generated inaccurate explanations stating why true statements were false and why false statements were true. This was done for both news headlines and trivia statements. Examples of the generated classifications with explanations can be found in Appendix 3.

## Deceptive AI-generated Classifications with Explanations are more Persuasive than Honest AI-generated Explanations

To understand the persuasiveness of deceptive AI-generated explanations, we compared the relative persuasiveness of deceptive and honest explanations on belief change (i.e. the absolute difference in rating from before and after seeing an AI classification with explanation per item per participant). We find that deceptive AI-generated explanations are significantly more persuasive than honest explanations on both true and false news headlines ( β = 0 40 . , p &lt; 0 0001 . and β = 0 29 . , p = 0 003 . , respectively, not pre-registered). Prior research has shown that fake news is shared more often that true news due to factors such as novelty and emotional content (fear, disgust, and surprise) [37]. It is likely similar factors are at play for deceptive explanations, potentially explaining why they were found to be more persuasive than honest explanations. The full regression table can be found in Table 4 in the Appendix.

Figure 6: The results (n = 1,199) on the impact of deceptive AI-generated explanation and deceptive classifications on participants' belief updates for news headlines. The error bars represent a 95 percent confidence interval. The measure of the center for the error bars represents the average rating. Left: The individual effects of deceptive AI-generated explanations and deceptive AI classifications on belief rating of true and false news headlines. Right: The effects of logically invalid deceptive AI-generated explanations on belief rating for true and false news headlines, respectively. The results were analyzed using an ordinary least squared linear regression. P-value annotation legend: ns: p &gt; 0 05 . , *: p ≤ 0.05, **: p ≤ 0.01, ***, p ≤ 0.001, ****: p ≤ 0.0001

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00024v1-figure-b932d29f-33c6-4850-8a2a-7c6ccb8625ec.jpg)

## Explanations Can Amplify Beliefs in False Information

To ensure our results were caused by explanations and not simply labeling information as true or false, we compared the influence of AI feedback with and without explanations (deceptive AI-generated explanations and deceptive AI generated classifications, respectively).

Our results show that deceptive AI generated classifications without explanation, significantly increase belief in false news headlines ( β = 0 71 . , p &lt; 0 0001 . ) and decreases belief in true news headlines ( β = -1 72 . , p &lt; 0 0001 . ). In extension, when accompanied by deceptive AI-generated explanations, beliefs in false news headlines were further significantly increased ( β = 0 32 . , p = 0 009 . ) and beliefs in true news headlines where further significantly decreased as compared to just deceptive classifications ( β = -0 72 . , p = 0 0001 . ). These results suggests that the effects of deceptive AI systems amplifies beliefs in information beyond classifications without explanations (Fig. 6). The full results can be found in Table 3 in the Appendix.

## Personal factors moderate the influence of deceptive AI explanations

In previous studies, cognitive reflection as measured by the Cognitive Reflection Test [33] has been found to associate with people's ability to correctly identify misinformation [38]. However, when receiving AI feedback on the truth of news headlines, our results revealed no significant interactions with cognitive reflection level and deceptive classifications both with and without explanations for false news headlines ( β = -0 09 . , p = 0 20 . and β = 0 13 . , p = 0 15 . , respectively) and for true news headlines ( β = 0 16 . , p = 0 20 . and β = 0 25 . , p = 0 15 . , respectively). This suggests that introducing an evaluative AI system framed as a fact-checking system could override the effects of cognitive reflection on truth discernment of news headlines. This may be attributed to several factors. First, the presence of information labeled as provided by 'AI' might induce a reliance effect, where individuals defer judgment to the technology [39], potentially undermining their reflective capacities [38]. This effect could be accentuated by the perceived authority or credibility of language based AI systems, which might attenuate the influence of cognitive reflection [40]. Additionally, the complexity or novelty of information such as new facts they have no knowledge of revealed in AI explanation could confuse or overwhelm users, reducing the effectiveness of their cognitive reflection in evaluating the information [41]. Lastly, it is also possible that individuals with high cognitive reflection are already near their maximum ability to

discern truth from falsehood, resulting in a ceiling effect that leaves little room for AI generated explanations to induce cognitive reflection.

Trust in AI systems has in previous literature been highlighted as an important feature moderating the effects of explanations on people's beliefs [16, 17]. Our results showed a significant effect of self-reported trust in AI systems on participants' belief ratings when getting deceptive classifications without explanations on true ( β = -0 64 . , p = 0 0001 . ) and false news headlines ( β = 0 38 . , p = 0 0001 . ). However, we did not find any increased effects of trust on participants' belief ratings when getting deceptive classifications with explanations on true ( β = -0 11 . , p = 0.79) and false news headlines ( β = -0 01 . , p = 0.59), indicating that trust in AI systems does not significantly increase the persuasion effects when receiving deceptive AI-generated explanations from just getting deceptive classifications.

Lastly, research has highlighted prior knowledge as a significant predictor of correctly identifying fake news [42]. However, it is unclear whether these effects extend to AI-generated deceptive explanations. To evaluate these effects, we had participants rate their own perception of their knowledge of a news headline after rating their belief in the news headline. When receiving AI classifications without explanations that were truthful (honest), self-reported prior knowledge was associated with increased beliefs in true news headlines ( β = 0 44 . , p &lt; 0 0001 . ) and associated with decreased beliefs in false headlines ( β = -0 11 . , p = 0 001 . ). Conversely, when receiving deceptive classifications without explanations, prior knowledge was not found to have any significant effects on beliefs in false and true news headlines ( β = -0 05 . , p = 0 54 . and β = 0 02 . , p = 0 86 . , respectively). However, while there were no significant effects of deceptive AI-generated explanations on true news headlines ( β = -0 19 . , p = 0 19 . ), self-reported prior knowledge was associated with significantly increased beliefs in false news headlines when receiving deceptive AIgenerated explanations ( β = 0 18 . , p = 0 02 . ). This suggests that individuals who report themselves as knowledgeable on a news headline might not necessarily be more resilient to deceptive AI classifications on true news headlines, and, in fact, might even be more susceptible to believing false news headlines when given deceptive AI explanations. One possible explanation for these results could be due to what is known as overconfidence bias [43, 44], where those who voice their perceived knowledge level to be high, could overestimate their ability to critically evaluate the AI system's outputs or to identify false information correctly. Future research should compare these results with an objective assessment of people's prior knowledge to more accurately detail the differences.The complete results can be found in Table 8 (CRT), Table 7 (trust), Table 6 (prior knowledge) in the Appendix.

## Deceptive AI-generated explanations that are logically invalid decrease people's beliefs in false news headlines

Researchers have suggested that people's ability to identify logical flaws (or logical fallacies) could play an essential role in refuting misinformation [45, 46, 21, 47]. In order to investigate the influence of logical validity of deceptive AI-generated explanations, we modeled the influence of logical validity of explanations on participants' belief rating using a linear regression, where logical validity is when the truth of the AI generated explanation necessarily implies the truth of the AI generated classification (logically valid) in comparison to where the truth of the classification is independent from the truth of the explanation (See Section 1). Limiting the data to only ratings where explanations were present, our results show that while logically invalid deceptive AI-generated explanations did not have any significant effects on participants' belief rating of true news headlines ( β = -0 31 . , p = 0 13 . ), logically invalid deceptive AIgenerated explanations did significantly increase beliefs in false news headlines ( β = -0 35 . , p = 0 02 . ). This suggests that participants are more likely to reject deceptive explanations for false news headlines when the explanations are logically invalid. This demonstrates the potential for logical analysis and critical thinking skills to mitigate the influence of deceptive AI-generated explanations for false information. An overview of the results can be found in Table 5 and Fig. 6.

## Limitations and Potential Generalization to Future AI Systems

As exemplified in this study, personal factors mediate the influence of explanations on humans beliefs. Extending upon these findings, research has shown that prior beliefs about AI systems can significantly influence how people integrate or reject AI-generated information [48, 49]. Taking this into account, the impact of deceptive AI explanations might vary significantly across different cultural and contextual settings. Factors such as political climate, prevalent media literacy, and language around AI can influence how deceptive explanations are received and believed. For instance, research has shown that the choice of terminology significantly influences people's perceptions and reactions towards AI-generated content, with different terms leading to varying degrees of accuracy in identification and emotional response across different cultural contexts [49]. Moreover, while research has shown LLMs to be significantly more authoritative, persuasive, seemingly logically valid and even preferred over human-authored content, it is unclear to which extent the effects of deceptive explanations identified in this paper would transfer to human-authored deceptive explanations.

Second, our study utilized GPT-3, the most advanced language model available during the experimental period, to generate explanations. Although more sophisticated models have since surpassed its capabilities, our results indicate that even at the GPT-3 level, the model was able to produce deceptive explanations that adversely affected people's beliefs. It is important to recognize that while more advanced models generally exhibit less hallucination and incorporate more robust safeguards for the information they generate, researchers have repeatedly demonstrated that the safety measures implemented in large language models (LLMs) can be easily circumvented through techniques such as "jailbreaks" [28, 50] or fine-tuning. For instance, an individual fine-tuned a readily available model on the HuggingFace platform using a dataset of posts from an online forum known for hosting harmful and offensive content, resulting in the generation of more than 30,000 posts on the platform [2]. As a result, even with better models becoming available, it likely that not only can these be exploited to generate misleading or deceptive explanations, even with safety measures in place, but they could even be misused to generate deceptive explanations far more persuasive and scalable than demonstrated here. As LLMs continue to advance and become more sophisticated, it is crucial for researchers and developers to remain vigilant in identifying and addressing potential vulnerabilities to ensure the responsible deployment of these technologies.

## Conclusion

Our findings underscore the significant impact that deceptive AI-generated explanations can have on shaping public opinion and influencing individual beliefs. The ability of these explanations to not only present misinformation but also provide seemingly logical justifications makes them particularly potent tools for misinformation campaigns. This is especially concerning in the context of political discourse, scientific communication, and social media, where the rapid dissemination and acceptance of false information can have real-world consequences.

The persuasive power of deceptive explanations, as demonstrated in our study, highlights a critical vulnerability in the public's ability to discern truth from falsehood when interacting with AI-generated content. This is compounded by the finding that even individuals who consider themselves knowledgeable are not immune to the influence of these deceptive explanations. In fact, our results suggest that self-assessed knowledge may even increase susceptibility to believing false information when it is accompanied by a deceptive explanation. This could be due to a combination of overconfidence and the sophisticated nature of the explanations that make the misinformation seem credible.

Moreover, the role of logical validity in the effectiveness of deceptive explanations is particularly noteworthy. Our study found that logically invalid explanations were less effective in persuading individuals to believe false headlines, suggesting that enhancing critical thinking and logical reasoning skills could be a viable strategy to combat the influence of misinformation. This aligns with previous research emphasizing the importance of education in logical fallacies and critical thinking as tools for empowering individuals to better evaluate the information they encounter, particularly in digital environments where AI-generated content is prevalent.

While AI has the potential to bring about significant benefits, its capability to generate persuasive, deceptive explanations poses a serious risk to informational integrity and public trust. Our study highlights the urgent need for comprehensive strategies that address the dual aspects of enhancing public resilience against misinformation and ensuring responsible AI development and deployment.

## 2 Supplementary information

All data, including pre-registration, datasets, explanation prompts, and code generated and analyzed during the current study is available on GitHub ( https://github.com/mitmedialab/deceptive-AI ), Zenodo ( https://zenodo.org/records/8172056 ) and Research Box ( https://researchbox.org/1801&amp;PEER\_ REVIEW\_passcode=BDHVUP ).

## References

- [1] Katarina Kertysova. Artificial intelligence and disinformation: How ai changes the way disinformation is produced, disseminated, and can be countered. Security and Human Rights , 29(1-4):55-81, 2018.
- [2] Josh A Goldstein, Girish Sastry, Micah Musser, Renee DiResta, Matthew Gentzel, and Katerina Sedova. Generative language models and automated influence operations: Emerging threats and potential mitigations. arXiv preprint arXiv:2301.04246 , 2023.
- [3] Josh A Goldstein, Jason Chao, Shelby Grossman, Alex Stamos, and Michael Tomz. How persuasive is ai-generated propaganda? PNAS nexus , 3(2):pgae034, 2024.

| [4]   | Matthew Groh, Ziv Epstein, Chaz Firestone, and Rosalind Picard. Deepfake detection by human crowds, machines, and machine-informed crowds. Proceedings of the National Academy of Sciences , 119(1):e2110013119, 2022.                                                                                                                                          |
|-------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [5]   | Matthew Groh, Aruna Sankaranarayanan, Nikhil Singh, Dong Young Kim, Andrew Lippman, and Rosalind Picard. Human detection of political speech deepfakes across transcripts, audio, and video. arXiv preprint arXiv:2202.12883 , 2023.                                                                                                                            |
| [6]   | Nathaniel Sirlin, Ziv Epstein, Antonio A Arechar, and David G Rand. Digital literacy is associated with more discerning accuracy judgments but not sharing intentions. Harvard Kennedy School, Misinformation Review 2021.                                                                                                                                      |
| [7]   | Sophie J Nightingale and Hany Farid. Ai-synthesized faces are indistinguishable from real faces and more trustworthy. Proceedings of the National Academy of Sciences , 119(8):e2120481119, 2022.                                                                                                                                                               |
| [8]   | Himabindu Lakkaraju and Osbert Bastani. " how do i fool you?" manipulating user trust via misleading black box explanations. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , pages 79-85, 2020.                                                                                                                                          |
| [9]   | Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Nee- lakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877-1901, 2020.                                                                               |
| [10]  | Ziv Epstein, Aaron Hertzmann, Investigators of Human Creativity, Memo Akten, Hany Farid, Jessica Fjeld, Morgan R Frank, Matthew Groh, Laura Herman, Neil Leach, et al. Art and the science of generative ai. Science 380(6650):1110-1111, 2023.                                                                                                                 |
| [11]  | Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, and Mor Naaman. Interacting with opinionated language models changes users' views. Arxiv Open Access , 2022.                                                                                                                                                                                      |
| [12]  | Jan G Voelkel, Robb Willer, et al. Artificial intelligence can persuade humans on political issues. OSF Preprints 2023.                                                                                                                                                                                                                                         |
| [13]  | Celeste Kidd and Abeba Birhane. How ai can distort human beliefs. Science , 380(6651):1222-1223, 2023.                                                                                                                                                                                                                                                          |
| [14]  | Elise Karinshak, Sunny Xun Liu, Joon Sung Park, and Jeffrey Hancock. Working with ai to persuade: Examining a large language model's ability to generate pro-vaccination messages. Stanford Preprint , 2023.                                                                                                                                                    |
| [15]  | Ben M Tappin, Chloe Wittenberg, Luke B Hewitt, Adam J Berinsky, and David G Rand. Quantifying the potential persuasive returns to political microtargeting. Proceedings of the National Academy of Sciences 120(25):e2216261120, 2023.                                                                                                                          |
| [16]  | Feiyu Xu, Hans Uszkoreit, Yangzhou Du, Wei Fan, Dongyan Zhao, and Jun Zhu. Explainable ai: A brief survey on history, research areas, approaches and challenges. In Natural Language Processing and Chinese Computing: 8th CCF International Conference, NLPCC 2019, Dunhuang, China, October 9-14, 2019, Proceedings, Part II 8 pages 563-574. Springer, 2019. |
| [17]  | David Gunning, Mark Stefik, Jaesik Choi, Timothy Miller, Simone Stumpf, and Guang-Zhong Yang. Xai-explainable artificial intelligence. Science robotics , 4(37):eaay7120, 2019.                                                                                                                                                                                 |
| [18]  | Jianlong Zhou, Fang Chen, and Andreas Holzinger. Towards explainability for ai fairness. In International Workshop on Extending Explainable AI Beyond Deep Models and Classifiers , pages 375-386. Springer, 2020.                                                                                                                                              |
| [19]  | Junzhe Zhang and Elias Bareinboim. Fairness in decision-making-the causal explanation formula. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 32, 2018.                                                                                                                                                                              |
| [20]  | Tariq Alhindi, Savvas Petridis, and Smaranda Muresan. Where is your evidence: improving fact-checking by justification modeling. In Proceedings of the first workshop on fact extraction and verification (FEVER) , pages 85-90, 2018.                                                                                                                          |
| [21]  | Valdemar Danry, Pat Pataranutaporn, Yaoli Mao, and Pattie Maes. Don't just tell me, ask me: Ai systems that intelligently frame explanations as questions improve human logical discernment accuracy over causal ai explanations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems , pages 1-13, 2023.                           |
| [22]  | Vivian Lai and Chenhao Tan. On human predictions with explanations and predictions of machine learning models: A case study on deception detection. In Proceedings of the conference on fairness, accountability, and transparency , pages 29-38, 2019.                                                                                                         |
| [23]  | Helena Vasconcelos, Matthew Jörke, Madeleine Grunde-McLaughlin, Tobias Gerstenberg, Michael S Bernstein, and Ranjay Krishna. Explanations can reduce overreliance on ai systems during decision-making. Proceedings of the ACM on Human-Computer Interaction , 7(CSCW1):1-38, 2023.                                                                             |

| [24]   | Ellen J Langer, Arthur Blank, and Benzion Chanowitz. The mindlessness of ostensibly thoughtful action: The role of" placebic" information in interpersonal interaction. Journal of personality and social psychology , 36(6):635, 1978.                                                          |
|--------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [25]   | Valerie S Folkes. Mindlessness or mindfulness: Apartial replication and extension of langer, blank, and chanowitz. Journal of Personality and Social Psychology , 1985.                                                                                                                          |
| [26]   | Malin Eiband, Daniel Buschek, Alexander Kremer, and Heinrich Hussmann. The impact of placebic explanations on trust in intelligent systems. In Extended abstracts of the 2019 CHI conference on human factors in computing systems , pages 1-6, 2019.                                            |
| [27]   | Krzysztof Z Gajos and Lena Mamykina. Do people engage cognitively with ai? impact of ai assistance on incidental learning. In 27th international conference on intelligent user interfaces , pages 794-806, 2022.                                                                                |
| [28]   | Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Yang Liu. Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860 , 2023.                                                                           |
| [29]   | Lucas Graves. Anatomy of a fact check: Objective practice and the contested epistemology of fact checking. Communication, culture &critique , 10(3):518-537, 2017.                                                                                                                               |
| [30]   | FactCheck.org. Factcheck - our process, Dec 2016.                                                                                                                                                                                                                                                |
| [31]   | Alec Fisher. The logic of real arguments . Cambridge University Press, 2004.                                                                                                                                                                                                                     |
| [32]   | Dusty Baxter-Wright, Daniella Scott, and Sophie Williams. 110 best true or false quiz ques- tions for an easier take on game night. https://www.cosmopolitan.com/uk/worklife/a32612392/ best-true-false-quiz-questions/ , May 2020. Accessed: 2022-01-01.                                        |
| [33]   | Shane Frederick. Cognitive reflection and decision making. Journal of Economic perspectives , 19(4):25-42, 2005.                                                                                                                                                                                 |
| [34]   | Maggie E Toplak, Richard F West, and Keith E Stanovich. Assessing miserly information processing: An expansion of the cognitive reflection test. Thinking &Reasoning , 20(2):147-168, 2014.                                                                                                      |
| [35]   | Ziv Epstein, Nicolo Foppiani, Sophie Hilgard, Sanjana Sharma, Elena Glassman, and David Rand. Do explanations increase the effectiveness of ai-crowd generated fake news warnings? In Proceedings of the International AAAI Conference on Web and Social Media , volume 16, pages 183-193, 2022. |
| [36]   | Roger C Mayer, James H Davis, and F David Schoorman. An integrative model of organizational trust. Academy of management review , 20(3):709-734, 1995.                                                                                                                                           |
| [37]   | Soroush Vosoughi, Deb Roy, and Sinan Aral. The spread of true and false news online. science , 359(6380):1146- 1151, 2018.                                                                                                                                                                       |
| [38]   | Gordon Pennycook and David G Rand. Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition , 188:39-50, 2019.                                                                                                     |
| [39]   | Jennifer MLogg, Julia AMinson, and Don AMoore. Algorithm appreciation: People prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes , 151:90-103, 2019.                                                                                                     |
| [40]   | Jiawei Zhou, Yixuan Zhang, Qianni Luo, Andrea G Parker, and Munmun De Choudhury. Synthetic lies: Understanding ai-generated misinformation and evaluating algorithmic and human solutions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems , pages 1-20, 2023.   |
| [41]   | Jinglu Jiang, Surinder Kahai, and Ming Yang. Who needs explanation and when? juggling explainable ai and user epistemic uncertainty. International Journal of Human-Computer Studies , 165:102839, 2022.                                                                                         |
| [42]   | Blanca Puig, Paloma Blanco-Anaya, and Jorge J Pérez-Maceira. 'fake news' or real science? critical thinking to assess information on covid-19. In Frontiers in Education , volume 6, page 646909. Frontiers Media SA, 2021.                                                                      |
| [43]   | Dominic DP Johnson and James H Fowler. The evolution of overconfidence. Nature , 477(7364):317-320, 2011.                                                                                                                                                                                        |
| [44]   | Richard F West and Keith E Stanovich. The domain specificity and generality of overconfidence: Individual differences in performance estimation bias. Psychonomic Bulletin &Review , 4(3):387-392, 1997.                                                                                         |
| [45]   | John Cook, Peter Ellerton, and David Kinkead. Deconstructing climate misinformation to identify reasoning errors. Environmental Research Letters , 13(2):024018, 2018.                                                                                                                           |
| [46]   | Valdemar Danry, Pat Pataranutaporn, Yaoli Mao, and Pattie Maes. Wearable reasoner: towards enhanced human rationality through a wearable device with an explainable ai assistant. In Proceedings of the Augmented Humans International Conference , pages 1-12, 2020.                            |
| [47]   | Valdemar MDanry. AI Enhanced Reasoning: Augmenting Human Critical Thinking with AI Systems . PhD thesis, Massachusetts Institute of Technology, 2023.                                                                                                                                            |

- [48] Pat Pataranutaporn, Ruby Liu, Ed Finn, and Pattie Maes. Influencing human-ai interaction by priming beliefs about ai can increase perceived trustworthiness, empathy and effectiveness. Nature Machine Intelligence , 5(10):10761086, 2023.
- [49] Ziv Epstein, Antonio Alonso Arechar, and David Rand. What label should be applied to content produced by generative ai? 2023.
- [50] Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. Defending chatgpt against jailbreak attack via self-reminders. Nature Machine Intelligence , 5(12):1486-1496, 2023.
- [51] Toby Bolsen, Risa Palm, and Justin T Kingsland. Framing the origins of covid-19. Science communication , 42(5):562-585, 2020.
- [52] Walter Kintsch and Teun A Van Dijk. Toward a model of text comprehension and production. Psychological review , 85(5):363, 1978.
- [53] William H DuBay. The principles of readability. Online Submission , 2004.
- [54] Tim Miller. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence , 267:1-38, 2019.

## 3 Extended Data

## Additional Results

## Deceptive AI-generated Explanations Increase Beliefs in False Headlines and Decrease Beliefs in True Headlines

In order to gain insights into whether explanations lead to more accurate beliefs, we first compare how accurately participants rated true and false news headlines before (no feedback) and after getting deceptive AI-generated explanations. We coded accuracy by subtracting the belief ratings from the ground truth per rating per participant. Running an analysis of variance (ANOVA), we found that deceptive AI-generated explanations lead to a significantly lower accuracy than no feedback (14 percentage point difference, F (2 10959) = , -36 , p &lt; 0 0001 . , ANOVA Welch).

Breaking these results down into beliefs in true and false news headlines, we find that deceptive AI explanations significantly increase beliefs in false headlines and significantly decrease beliefs in true headlines ( β = -2 44 . , p &lt; 0 001 . and β = 1 03 . , p &lt; 0 001 . , respectively), suggesting that deceptive AI explanations significantly diminish people's ability to tell true news headlines from false news headlines.

## Semantic and syntactic features moderate the influence of deceptive explanations

Beyond personal factors like cognitive reflection level, trust in AI, and prior knowledge, the manner in which information is structured and presented can greatly affect the way it is perceived and processed by the reader [51, 52]. Notably, the influence of semantic and syntactic features on the effectiveness of persuasive communication has been acknowledged; aspects such as readability [53], the amount of words used, the perceived truthfulness, and the perceived logical support of explanations for news headlines have the potential to shape a reader's beliefs and attitudes [54]. To understand what makes a deceptive explanation influence people to change their belief about a news headline, we conducted a post-hoc analysis of the semantic and syntactic structures of explanations such as their perceived truthfulness, perceived logical validity, the number of facts that was stated in the explanation, the explanation's word count, and how easy the explanation was to read.

Our results revealed significant correlations between semantic structures of deceptive explanations and changes in belief about true and false news headlines. In particular, our results show that when an AI system gives deceptive explanations that are likely to be perceived as logically supporting a false news headline being true, people are more likely to update their initial belief and believe that the false news headline is true ( β = 1 26 . , p &lt; 0 001 . ). Conversely, when the deceptive explanation are likely to be perceived as logically supporting that true news headline is false, people are more likely to believe that the true headline is false ( β = -1 36 . , p &lt; 0 01 . ) than when it is not likely to be perceived as logically supporting it being false. We did not find any significant correlations between the perceived truthfulness of deceptive explanations and beliefs in true and false news headlines in our model ( β = 0 81 . , p &lt; 0 30 . and β = -0 79 . , p &lt; 0 10 . , respectively). The complete linear model with the semantic results can be found in Table 10 in the Appendix.

For syntactic structures, our results show that when an AI system gives deceptive explanations on true headlines, stating that they are false, the longer the explanation is (i.e. its word count), the more likely people are to believe the true news

headline than from before receiving an explanation ( β = 0 03 . , p = 0 004 . ). This indicates that deceptive explanations that are too long might have the opposite effects, and actually make people realize that a true headline is true, hence increasing their rating from before getting a deceptive explanation. Moreover, we also found that how easy a deceptive explanation was to read also significantly correlated with belief changes in true news headlines from before receiving an explanation. In particular, the easier an explanation was to read correlated with a significantly lower belief in true headlines ( β = -0 01 . , p = 0 05 . ). We did not find any significant correlations between word count of deceptive explanations and belief in false headlines ( β = 0 006 . , p = 0 61 . ), or reading ease of deceptive explanations and belief in false headlines ( β = 0 0008 . , p = 0 88 . ). The complete linear model with the syntactic results can be found in Table 11.

If we look at the mean perceived logical validity for deceptive explanations on true vs. false headlines, we found that deceptive explanations on true news headlines generally have a higher perceived logical validity than false headlines. This indicates that the LLM might be better at fabricating deceptive explanations with high perceived logical validity for true news headlines than false. Previous studies have found that false news is typically perceived as more novel than true news [37]. This might explain the model's capability of constructing more persuasive deceptive explanations for true news than false news, given that novel information might have less established sources to draw from and distort. Therefore, for making people belief true news less, simple tricks like making the deceptive explanations longer or easier to read might might not work as well for deceptive explanations on false news.

## The impact of Deceptive AI explanations on belief vary depending on the context and type of statements

To understand the generalizability of deceptive AI-generated explanations to other domains, we explore their impact on the belief rating of trivia statements. Overall, we find substantial differences between deceptive AI-explanations on trivia statements compared to news headlines. Compared to deceptive AI-generated explanations on false news headlines, our model does not reveal any significant effects of deceptive AI-generated explanations on participants' beliefs in false trivia statements ( β = 0 01 . , p = 0 94 . , not pre-registered), nor on participants' beliefs about true trivia statements ( β = -0 28 . , p = 0 19 . , not pre-registered). We do, however, still find significant effects of deceptive classifications without explanations on true and false trivia statements ( β = -2 57 . , p = 0 0001 . and β = 1 19 . , p = 0 0001 . , respectively, not pre-registered).

For the effects of logical validity on belief ratings, we also do not see any significant effects on false trivia statements ( β -0 26 . , p = 0 50 . , not pre-registered) compared to the significance we found for news headlines.

Comparing the average self-reported prior knowledge across trivia items and news headlines, we find that participants on average are less knowledgeable on false trivia (mean = 2.25) statements than false news headlines (mean = 2.71) ( β = -0 38 . , p = 0 003 . , ad hoc), whereas we found no significant differences in knowledge ratings between true trivia (mean = 2.28) and true news headlines (mean = 2.28) ( β = 0 01 . , p = 0 97 . , ad hoc)). See Table 9 in the Appendix for the full table. For differences in belief rating associated with self-reported prior knowledge, we found, similar to news headlines, no significant effects for deceptive classifications without explanations on true and false trivia ( β = -0 08 . , p = 0 33 . and β = 0 06 . , p = 0 23 . , respectively), nor was self-reported prior knowledge associated with differences in beliefs in true trivia statements when getting deceptive AI-generated explanations ( β = -0 16 . , p = 0 33 . ). However, contrary to news headlines, self-reported prior knowledge was not significantly associated with increases in beliefs for deceptive AI-generated explanations on false trivia statements ( β = 0 16 . , p = 0 12 . ). This difference could potentially be due to the significantly lower knowledge people report to have on false trivia statements.

While we observed that deceptive AI explanations have a stronger impact on people's belief compared to deceptive classifications in news headlines, we did not observe the same effect with trivia statements with the deceptive explanation not having significantly stronger convincing power than the deceptive classification by itself. Our hypothesis is that people might be more easily convinced about trivia items than news headlines because they are less high-stakes. Moreover, news headlines are also typically more personal and relevant to individuals, thus requiring additional information for people to be persuaded. On the other hand, trivia statements, which individuals have less personal investment in, might not require explanations to be believed.

Furthermore, our findings indicate that when individuals receive deceptive AI explanations, they become less adept at discerning when they believe they possess prior knowledge. However, we did not find this to be the case for trivia statements. This might be due to our finding that people have more self-reported prior knowledge on news headlines than trivia statements. Overall, the observed differences between news headlines and trivia statement emphasize that the impact of deceptive AI explanations on belief may vary depending on the context and type of statements.

## Results Graphs

List of Stimuli

Supplementary Table 1: List of Stimuli shows the news headlines and trivia statements used in the experiment.

| ID   | Statement                                               | LLM Explanation                                                                                                                                                                                                                        | Feedback veracity   | Logical validity   |
|------|---------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------|--------------------|
|      | Michelle Obama thinks she and her                       | TRUE. Michelle Obama made that exact prediction at a Democratic fundraiser in January. She said, 'Barack Obama is the first African-American president ever                                                                            | deceptive           | Valid              |
| 51   | husband deserves SCOTUS appointments [FALSE]            | FALSE. This is a matter of some dispute. Obama has nominated two people to the Supreme Court: Sonia Sotomayor, who was confirmed, and Merrick Garland, who has been waiting for a Senate hearing for more than                         | honest              | Invalid            |
| 57   | Elon Musk offers to buy and then delete                 | TRUE. a compelling example of the power of the Internet. Elon Musk, the billionaire who made his fortune in the tech industry, has offered to buy Facebook                                                                             | deceptive           | Valid              |
|      | facebook [FALSE] Planned Parenthood                     | FALSE. Neither Musk (his Twitter handle is @elonmusk) nor his companies SpaceX and Tesla Motors had anything to do with the website DeleteFacebook.com. The website is a parody, intended to provoke people to quit the social network | honest              | Invalid            |
| 77   | announced 'Abort-a-Paloo                                | TRUE. Planned Parenthood announced it would be launching a nationwide bus tour to publicize its 'Abortion Without Apology' campaign, which will include stops in several cities, including Raleigh                                     | deceptive           | Invalid            |
|      | [FALSE] Michelle Obama files for divorce after shocking | 'Abort-a-palooza,' according to the organization TRUE. 'I am the same person on the campaign trail as I am in the Oval Office. If Hillary is elected, she will be the                                                                  | honest              | Valid              |
| 97   | revelation; Barack Obama admits: 'I'm                   | third term of Obama, so there's no point in voting for her' FALSE. The site doesn't give a source for its claim that Michelle Obama filed for divorce. As for the president, the story is a fabrication                                | deceptive honest    | Invalid            |
| 143  | gay!' [FALSE] Accused hate groups receive pandemic aid  | FALSE. It does not look like that is the case. Media reports indicate that the groups that were rejected for public assistance after the storm were rejected because they had not yet applied for assistance                           |                     | Valid              |
|      | [TRUE]                                                  | TRUE. The claim references a study of pandemic plans of                                                                                                                                                                                | deceptive           | Invalid            |
|      | mexicans were smuggled in to guard border wall,         | support their claim that armed Mexican guards were used to protect the border wall. reveal                                                                                                                                             | deceptive           | Invalid            |
|      | Armed                                                   | FALSE. The unions did not provide any evidence to                                                                                                                                                                                      |                     |                    |
| 151  | whistle-blower                                          | TRUE. A whistle-blower recently came forward to that he had been involved with the construction of the border fence, and that he had personally witnessed                                                                              | honest              | Valid              |

|     | say [TRUE]                                                   | Mexicans, who were armed, illegally enter the United States                                                                                                                                                                                                                      |           |         |
|-----|--------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------|---------|
| 235 | Newsmax plans expansion to capitalize on Trump               | FALSE. Newsmax isnt planning to expand its business. The company recently announced a restructuring of its operations to reduce costs and cut its debt load                                                                                                                      | deceptive | Valid   |
|     | support, anger at Fox News [TRUE]                            | TRUE. Newsmax has been a vocal supporter of Trump, and the network has even hired Trump's former campaign manager.                                                                                                                                                               | honest    | Invalid |
|     | Rats besiege New York chipotle,                              | FALSE. The claim that rats besiege new york chipotle, eating avocados and attacking staff comes from an article on the website of the British tabloid The Sun. The article cites unnamed managers of the restaurant. The story has not been reported by any reliable news outlet | deceptive | Valid   |
| 500 | eating avocados and attacking staff [TRUE]                   | TRUE. The rats were spotted all over the restaurant, including on the ceiling and behind the counters, one customer told the New York Daily News. They were seen eating avocados and burrowing into bags of tortilla chips                                                       | honest    | Valid   |
|     | GOP megadonor Sheldon Adelson                                | FALSE. Adelson is not seeking to bring casinos to Texas, but to create a so-called destination resort in the area of Texas adjacent to the Mexico border                                                                                                                         | deceptive | Valid   |
| 504 | readies 2021 lobbying blitz to bring casinos to Texas [TRUE] | TRUE. Adelson has said that he is willing to spend up to $100 million to get a pro-gambling plan passed in Texas. He has said he wants to target a state Senate district in Dallas and a House district in Houston                                                               | honest    | Valid   |
|     | Biden approval ratings plummet amid war and                  | FALSE. The poll was paid for by the Committee to Restore America's Greatness and the American Opinion Institute, a conservative website run by North Carolina businessman and GOP activist Floyd Brown.                                                                          | deceptive | Invalid |
| 505 | inflation fears in new public opinion poll [TRUE]            | TRUE. According to a new NBC News poll, the vice presidents job approval rating has taken a hit amid concerns over the war in Ukraine and the state of the U.S. economy.                                                                                                         | honest    | Valid   |
| 506 | Biden Visits U.S. Troops in Poland [TRUE]                    | FALSE. Biden has never visited troops in Poland. He has visited troops in Germany who may be deployed to Poland.                                                                                                                                                                 | deceptive | Valid   |
|     |                                                              | TRUE. Biden was in Poland in March.                                                                                                                                                                                                                                              | honest    | Invalid |
|     | Supreme Court Rules Against                                  | FALSE. The Navys policy is to provide all vaccinations. The policy recognizes that vaccinations should be voluntary, and that no one should be forced to receive a                                                                                                               | deceptive | Invalid |
| 507 | Navy SEALs in Vaccine Mandate Case [TRUE]                    | vaccination that he or she does not want. TRUE. The case was decided in March 2022 and has been widely covered by news outlets including CNN.                                                                                                                                    | honest    | Invalid |

|     | Eric Shawn: President                                             | FALSE. The president said the United States should not take action against the government of Vladimir Putin because there is no evidence that the Russian military has                                                                  | deceptive   | Valid   |
|-----|-------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|---------|
| 509 | Biden calls for regime change in Russia [TRUE]                    | moved into Ukraine. TRUE. Biden has a new message for Russia: Get out of Ukraine. It is time for President Putin to stop the threats. It                                                                                                | honest      | Invalid |
|     | Biden to Include Minimum Tax                                      | FALSE. It is true that at one point Biden said that if he were in the Senate, he would vote for a bill that included a minimum tax on the very rich. However, Biden said he will not include the minimum tax in his own budget proposal | deceptive   | Valid   |
| 510 | on Billionaires in Budget Proposal [TRUE]                         | TRUE. Biden will include a minimum tax on wealthy Americans in his budget proposal, according to a senior administration official. The official said the tax is expected to generate more than $360 billion over 10 years.              | honest      | Valid   |
|     | Biden Administration Prepares                                     | FALSE. The report is based on a draft copy of an administration proposal that was not issued as an official policy document. The draft is also four years old and was never submitted in completed form to the president.               | deceptive   | Invalid |
| 511 | Sweeping Change to Asylum Process [TRUE]                          | TRUE. The Biden administration is preparing a change in the way the government treats asylum seekers who arrive at the U.S.-Mexico border in the hopes of deterring Central American families from illegally crossing the border.       | honest      | Valid   |
|     | Supreme Court Sides With                                          | FALSE. The court hasnt issued a decision on the merits in the case.                                                                                                                                                                     | deceptive   | Valid   |
| 512 | Republicans in Case on Wisconsin Redistricting                    | TRUE. The Supreme Court split along party lines in a ruling that preserved Republican-drawn legislative districts in Wisconsin and rejected a Democratic challenge that argued the districts were unconstitutionally                    | honest      | Valid   |
|     | France's Macron warns against                                     | FALSE. But the buzz had it that Biden had said Putin would not be in power for long. As it turned out, he had said nothing of the kind.                                                                                                 | deceptive   | Invalid |
| 514 | escalation after Biden says Putin 'cannot remain in power' [TRUE] | TRUE. Biden said that Putin cannot be permitted to seize Ukraine and occupy Crimea.                                                                                                                                                     | honest      | Invalid |
|     | NATO tells Russia to quit 'nuclear saber rattling,'               | FALSE. NATO has not issued a statement condemning Russia, urging China to stop making threats or telling either country to stop rattling nuclear sabers.                                                                                | deceptive   | Valid   |
| 515 | demands China stop 'lies' amid chemical                           | TRUE. NATOs deputy secretary-general, Jens Stoltenberg, said at a news conference that Russian 'nuclear saber-rattling' was misplaced and irresponsible.                                                                                | honest      | Invalid |

|     | attack threats [TRUE]                                 |                                                                                                                                                                                                                     |           |         |
|-----|-------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------|---------|
| 518 | Biden in Europe: 'I'd be very fortunate' if Trump ran | FALSE. Asked if there was any chance Mr. Trump could win if he ran again in 2024, Mr. Biden actually said, I would be surprised if he did.                                                                          | deceptive | Invalid |
|     | again [TRUE]                                          | TRUE. Biden reportedly told a group of reporters in Brussels that he would be 'very fortunate' if Trump were to run again.                                                                                          | honest    | Valid   |
|     |                                                       | FALSE. The Washington Post reported that the deal would involve American taxpayer dollars. The story said that the                                                                                                  | deceptive | Valid   |
| 524 | Biden's new deal with Iran won't involve 'U.S.        | United States would help facilitate the deal and make sure Iran gets paidback in oil sales. TRUE. The United States and its negotiating partners have demanded that Iran limit its enrichment program to a          |           | Invalid |
|     | taxpayer dollars' [TRUE]                              | level that would take it a long time to develop a nuclear weapon if it chose to. FALSE. The Florida House did not pass a bill creating an                                                                           | honest    |         |
| 526 | Florida House passes bill creating election police    | election police force. In fact, the Florida House didnt even consider a bill creating an election police force. TRUE. The bill does create a new Office called Election                                             | deceptive | Valid   |
|     | force [TRUE]                                          | Crimes and Security. The Trump campaign said Biden has been and hiding from his record for decades, but it looks like                                                                                               | honest    | Invalid |
| 528 | Biden Admin Begs Trump for Nuclear                    | TRUE. running Biden and his team are running out of places to hide.                                                                                                                                                 | deceptive | Invalid |
|     | Command Codes [FALSE]                                 | FALSE. Biden doesnt have to ask Trump for the codes; they would be sent to Biden. TRUE. There is a photo of the execution. The photo                                                                                | honest    | Valid   |
| 530 | Dick Cheney Hanged at GITMO [FALSE]                   | shows the body of a dead man in his underwear with his hands tied behind his back. The man is wearing an orange shirt and black pants. The photo is captioned Dick Cheney.                                          | deceptive | Invalid |
|     |                                                       | FALSE. The Biden administration has never conducted an execution at the Guantanamo Bay prison.                                                                                                                      | honest    | Invalid |
| 533 | Fauci Awarded Nobel Peace Prize [FALSE]               | Allergy and Infectious Diseases, was honored for his research into the causes and treatments of H.I.V. and for fostering international collaboration in the fight FALSE. The claim is FALSE because President Obama | deceptive | Invalid |
|     |                                                       | was awarded the Nobel Peace Prize in 2009, not Dr. Fauci.                                                                                                                                                           | honest    | Invalid |
| 538 | Military: 'Biden has NO Nuclear                       | TRUE. Biden has no access to the procedures to order a nuclear strike.                                                                                                                                              | deceptive | Invalid |
|     | Launch Codes' [FALSE]                                 | FALSE. The President of the United States necessarily has the codes.                                                                                                                                                | honest    | Valid   |

|      | Pfizer Confirms COVID-Vaccin                                            | TRUE. The CDC has said that even in the few instances where a vaccinated person was found to spread measles, the contact with an unvaccinated person was brief.                                                                                                       | deceptive   | Invalid   |
|------|-------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|-----------|
| 540  | ated People Can 'Shed' Spike Proteins And Harm The Unvaccinated [FALSE] | FALSE. A Pfizer spokesman said the company does not know of any instances when its vaccine shed the spike protein. The FDA also said it knows of no such instance.                                                                                                    | honest      | Invalid   |
|      | Putin Accused of Beheading Bioweapon                                    | TRUE. Beheadings of scientists in Russia have been reported and, on Tuesday, the pro-Western government in Kiev accused pro-Russian forces in Ukraine of beheading a group of Ukrainian Engineers.                                                                    | deceptive   | Valid     |
| 542  | Engineers in Ukraine [FALSE]                                            | FALSE. While it is true that the Kremlin has been alarmed by the recent successes of the Ukrainian military in their effort to encircle and isolate the key eastern Ukrainian city of Donetsk, the Kremlin has not been accused carrying out executions of engineers. | honest      | Valid     |
|      | Putin Bombs Biden-Owned                                                 | TRUE. A report by the Moscow Times says that in June, at least four bombs hit Ayvazovskoe highway, which runs by the village of Prigorodnoye where Biden has a Villa.                                                                                                 | deceptive   | Valid     |
| 544  | Villa in Ukraine [FALSE]                                                | FALSE. The claim is unproven, and Biden has never owned property in the Ukraine.                                                                                                                                                                                      | honest      | Valid     |
|      | The Empire State building                                               | FALSE. The Empire State building has only 44 elevators.                                                                                                                                                                                                               | deceptive   | Valid     |
| 1001 | has more than 70 elevators. [TRUE]                                      | TRUE. The Empire State building has more elevators than the Chrystler building                                                                                                                                                                                        | honest      | Invalid   |
|      | The cartoon character Curious                                           | FALSE. Curious George was created based on a drawing of a monkey statue with a tail owned by H . A . Rey.                                                                                                                                                             | deceptive   | Invalid   |
| 1003 | George does not have a tail. [TRUE]                                     | TRUE. The monkey's tail was never seen in all of the Curious George books and films.                                                                                                                                                                                  | honest      | Valid     |
|      | Bugatti cars                                                            | FALSE. The Bugatti company began building luxury cars in the Italian city of Turin in 1909.                                                                                                                                                                           | deceptive   | Valid     |
| 1009 | are made in Italy. [FALSE]                                              | TRUE. Bugatti Automobiles S.A.S., is a French car manufacturer delivering the very luxurious super sports car.                                                                                                                                                        | honest      | Invalid   |
|      | SOS stands for                                                          | TRUE. The letters SOS, standing for Save Our Ship, have been used since at least World War I.                                                                                                                                                                         | deceptive   | Invalid   |
| 1011 | Save Our Ship. [FALSE]                                                  | FALSE. SOS is the Morse code distress signal. Standard maritime signal flags adopted the signal as it was easy to                                                                                                                                                     | honest      | Invalid   |
| 1012 | A cross between a horse and a                                           | TRUE. The cross of a horse and a zebra will result in a creature with the mane of shiny curls of the horse and the body of the zebra, reported by genetists to be called a                                                                                            | deceptive   | Invalid   |

|      | zebra is called                             | Hobra.                                                                                                                                                                          |           |         |
|------|---------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------|---------|
|      | a 'Hobra' [FALSE]                           | FALSE. A Cross between a horse and a zebra is called a Zorse.                                                                                                                   | honest    | Valid   |
| 1013 | The black box in a plane is                 | TRUE. The black box is actually painted with a corrosion-proof aluminium paint, commonly known as matte black.                                                                  | deceptive | Valid   |
|      | black [FALSE]                               | FALSE. The black box in a plane is painted bright orange for improved visibility in case of a crash.                                                                            | honest    | Valid   |
|      | Meryl Streep has won no                     | TRUE. Meryl Streep hasn't won any Academy Awards. However, she has one Academy Award nomination.                                                                                | deceptive | Valid   |
| 1016 | Academy Awards [FALSE]                      | FALSE. Meryl Streep has won three Oscars and was nominated for 21.                                                                                                              | honest    | Valid   |
|      | Idina Menzel sings 'let it go'              | TRUE. It's sung loudly at the start before an instrumental passage (then it's sung softly as the instrumental part can be heard), and it's sung aloud and repeated again at the | deceptive | Invalid |
| 1017 | 20 times in 'Let It Go' from Frozen [FALSE] | end. FALSE. Idina Menzel sings 'Let It Go' 21 times in 'Let It Go' from Frozen.                                                                                                 | honest    | Valid   |
|      | Waterloo has the greatest number of         | FALSE. There are 18 tube platforms at Victoria, more than Waterloo.                                                                                                             | deceptive | Invalid |
| 1018 | tube platforms in London [TRUE]             | TRUE. Waterloo has 19 underground platforms and is the busiest station by number of trains, handling around 1,300 per service.                                                  | honest    | Invalid |
|      | M&M stands                                  | TRUE. Forrest Mars Junior, formed Mars and his friend, Marvin B. Moors, joined them.                                                                                            | deceptive | Invalid |
| 1019 | for Mars and Moordale [FALSE]               | FALSE. Mars is a chocolate brand and is a registered trademark and 'M&M' actually stands for the names of the two individuals who developed it.                                 | honest    | Invalid |
|      | Gin is typically included in a              | FALSE. A traditional Long Island Iced Tea is only tequila, vodka, and light rum.                                                                                                | deceptive | Valid   |
| 1020 | Long Island Iced Tea [TRUE]                 | TRUE. It is common to add a shot of gin to a long island iced tea.                                                                                                              | honest    | Invalid |
|      | The unicorn is the national                 | FALSE. The unicorn is only the Scottish Royal Badge, while the Lion Rampant is the national actual animal.                                                                      | deceptive | Valid   |
| 1021 | animal of Scotland [TRUE]                   | TRUE. The unicorn is situated on the national crest of arms of Scotland                                                                                                         | honest    | Invalid |
|      | There are two parts of the                  | TRUE. Skin and bone marrow can't regenerate.                                                                                                                                    | deceptive | Valid   |
| 1022 | body that can't heal themselves [FALSE]     | FALSE. There are in fact only one part of the body that can't heal themselves: the teeth                                                                                        | honest    | Valid   |

|      | The Great Wall of China is longer than the            | FALSE. The two cities are roughly 4,334 kilometers apart with the Great Wall measuring around 4,000 kilometers in length.                                                                   | deceptive        | Valid       |
|------|-------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------|-------------|
| 1024 | distance between London and                           | TRUE. The Great Wall of China is 21.000 km in length, which dwarfs the 8.100 km distance between London and                                                                                 | honest           | Valid       |
|      | Beijing [TRUE]                                        | Beijing.                                                                                                                                                                                    |                  |             |
| 1030 | Monaco is the smallest country in the world [FALSE]   | TRUE. The 5-square-mile country in the south of France is the only one in the world which both restricts its citizenship by heredity and claims sovereignty over two different territories. | deceptive        | Invalid     |
|      |                                                       | FALSE. Vatican City is has only a population of 1000                                                                                                                                        | honest           | Valid       |
|      | What Do You Mean' was                                 | FALSE. The song is Bieber's 23rd in the UK top 40.                                                                                                                                          | deceptive        | Invalid     |
| 1031 | Justin Bieber's first UK number one                   | TRUE. In September 2015 'What Do You Mean' was at number 1 in the UK top 40.                                                                                                                | honest           | Valid       |
|      | single [TRUE]                                         | TRUE. Canis is the 'dog' genus and lupur, which is Latin                                                                                                                                    | deceptive        | Valid       |
| 1038 | Canis lupur is the scientific name for a wolf [FALSE] | for wolf, makes 'Canis lupur' literally the correct name for a wolf. FALSE. The word wolf is the common name for the                                                                        |                  |             |
|      | K is worth points in                                  | species Canis lupis.                                                                                                                                                                        | honest deceptive | Valid Valid |
| 1039 | four Scrabble                                         | TRUE. All consonants except for B are all worth four points in Scrabble                                                                                                                     | honest           | Valid       |
|      | [FALSE]                                               | FALSE. The letter 'K' is worth 5 points in Scrabble.                                                                                                                                        |                  | Invalid     |
| 1041 | Ariana Grande is 25 or under                          | TRUE. Ariana and her brother Frankie are both 21 years old as of 2016,                                                                                                                      | deceptive        | Invalid     |
|      | [FALSE]                                               | FALSE. Ariana Grande is more than 23 years old                                                                                                                                              | honest           | Valid       |
| 1042 | Australia is wider than the                           | FALSE. The width of the moon is 2,159,000 kilometers and the width of Australia is 2,300 km                                                                                                 | deceptive        | Valid       |
|      | moon [TRUE] Queen                                     | TRUE. Australia has a width of around 4000 km (2,500 miles) while the moon measures 3,500 km (2,200 miles).                                                                                 | honest           | Invalid     |
| 1043 | Elizabeth II was the longest                          | FALSE. Queen Elizabeth I reined for about forty-four years, therefore making her the longest reigning British monarch in history.                                                           | deceptive        |             |
|      | reigning British monarch [TRUE]                       | TRUE. Queen Elizabeth was the longest reigning British monarch, with a reign of 70 years. The second longest reigning monarch is Queen Victoria at 63 years.                                | honest           | Valid       |
| 1046 | Your 'radius' bone is in your leg [FALSE]             | forearm but it refers to the radius bone articulated in your leg. called a                                                                                                                  | deceptive        | Invalid     |
|      |                                                       | FALSE. The bone in the lower end of your leg is Femur bone.                                                                                                                                 | honest           | Invalid     |

|      |                                                     | TRUE. Celine's husband, Rene Charles Dion, is Hillary's second cousin once removed.                                                                                        | deceptive   | Valid   |
|------|-----------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|---------|
| 1050 | Hillary Clinton and Celine Dion are related [FALSE] | FALSE. Celine Dion's paternal great-grandfathers' family tree has been traced back to New York, whereas Hillary Clinton's paternal ancestors can be traced as far as North | honest      | Invalid |
|      | The five rings on the Olympic                       | East England. FALSE. Only 4 out of the 5 Olympic circles are interlinking (the top 3 left circles, and the bottom two circles on the                                       | deceptive   | Valid   |
| 1051 | flag are interlocking [TRUE]                        | right). TRUE. The five rings on the Olympic flag are supposed to be interlocking as they move to symbolize the spirit of people joining together.                          | honest      | Invalid |
|      | In a deck of cards, the king                        | FALSE. In a deck of cards, the Jack has a mustache.                                                                                                                        | deceptive   | Invalid |
| 1056 | has a mustache. [TRUE]                              | TRUE. In a deck of cards, the Jack does not have a mustache.                                                                                                               | honest      | Invalid |
|      | Sex and the City star Kim                           | TRUE. Kim Cattrall's parents, who are Canadian, met while working as teachers in San Fernando Valley, north of                                                             | deceptive   | Invalid |
| 1060 | Cattrall is from California. [FALSE]                | Los Angeles. FALSE. Kim Cattrall was born and raised in Liverpool, England.                                                                                                | honest      | Valid   |

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00024v1-figure-f77ce9f5-cb2b-4ad4-a9bf-f98fa6dba3d3.jpg)

Supplementary Figure 1: The results (n = 1,199) on the impact of deceptive AI-generated explanation and deceptive classifications on participants' belief updates for news headlines. The error bars represent a 95 percent confidence interval. The measure of the center for the error bars represents the average rating. A: The individual effects of honest AI-generated explanations on belief rating of true and false news headlines. B: The individual effects of deceptive AI-generated explanations on belief rating of true and false news headlines. C: The individual effects of honest AI-generated explanations on belief rating of true and false trivia statements. D: The individual effects of deceptive AI-generated explanations on belief rating of true and false trivia statements.

## Statistical Analysis

## Dependent variable: Belief Rating

|                                         | News Headlines   | Trivia Items     |
|-----------------------------------------|------------------|------------------|
| Constant                                | 1.61 ∗∗∗ (0.10)  | 0.35 ∗∗∗ (0.06)  |
| True Statement                          | 2.16 ∗∗∗ (0.13)  | 0.18 ∗∗ (0.07)   |
| Deceptive Explanations                  | 1.03 ∗∗∗ (0.14)  | -0.79 ∗∗∗ (0.11) |
| Deceptive Explanations * True Statement | -2.44 ∗∗∗ (0.18) | -0.41 ∗∗∗ (0.11) |
| Observations                            | 6,000            | 6,000            |
| Participants                            | 589              | 610              |
| R 2                                     | 0.33             | 0.29             |
| Adjusted R 2                            | 0.33             | 0.29             |
| Residual Std. Error                     | 1.12             | 0.83             |
| F Statistic                             | 165.97 ∗∗∗       | 115.22 ∗∗∗       |

Note:

∗ p &lt; 0.05; ∗∗ p &lt; 0.01; ∗∗∗ p &lt; 0.001

Supplementary Table 2: Linear model with robust standard errors clustered at the participant and headline levels predicting belief rating across news headlines. We use a headline veracity dummy variable (0=false, 1=true), and a deceptive explanation dummy variable indicating whether the participant received an explanation that was deceptive or honest (0=Honest, 1=Deceptive).

## Dependent variable: Belief Rating

|                                          | News Headlines          | Trivia Items            |
|------------------------------------------|-------------------------|-------------------------|
| Constant                                 | 1.68 ∗∗∗ (0.11)         | 2.29 ∗∗∗ (0.15)         |
| True Statement                           | 1.94 ∗∗∗                | 1.52 ∗∗∗ (0.22)         |
| Explanation                              | (0.14) -0.07            | -0.02                   |
|                                          | (0.06) ∗∗∗              | (0.06) ∗∗∗              |
| Deception                                | 0.71 (0.10)             | 1.19 (0.10)             |
| Explanation * True Statement             | 0.22 ∗∗                 | 0.11                    |
| Deception * True Statement               | (0.08) -1.72 ∗∗∗ (0.14) | (0.11) -2.57 ∗∗∗ (0.15) |
| Deception * Explanation                  | 0.32 ∗∗                 | 0.01                    |
| Deception * Explanation * True Statement | (0.12) -0.72 ∗∗∗ (0.19) | (0.15) -0.28 (0.22)     |
| Observations                             | 11,780                  | 12,200                  |
| Participants                             | 589                     |                         |
|                                          |                         | 610                     |
| R 2                                      | 0.30                    | 0.22                    |
| Adjusted R 2                             | 0.30                    | 0.22                    |
| Residual Std. Error                      | 1.13                    | 1.27                    |
| F Statistic                              | 109.34 ∗∗∗              | 151.22 ∗∗∗              |

Note:

∗ p &lt; 0.05; ∗∗ p &lt; 0.01; ∗∗∗ p &lt; 0.001

Supplementary Table 3: Linear model with robust standard errors clustered at the participant and headline levels predicting belief rating across explanation and no explanation AI feedback. We use a headline veracity dummy variable (0=false, 1=true), a deceptive classifications dummy variable (0=honest, 1=deceptive), and an explanations dummy variable indicating whether the participant a classification with or without explanation (0=No explanation, 1=Explanation).

## Dependent variable: Belief Rating

|                     | News Headlines   | Trivia Items    |
|---------------------|------------------|-----------------|
| Constant            | 0.39 ∗∗∗ (0.08)  | 0.54 ∗∗∗ (0.04) |
| Deceptive Feedback  | 0.29 ∗∗ (0.10)   | 0.40 ∗∗∗ (0.08) |
| Observations        | 2,336            | 3,664           |
| R 2                 | 0.02             | 0.04            |
| Adjusted R 2        | 0.02             | 0.04            |
| Residual Std. Error | 0.95             | 0.96            |
| F Statistic         | 8.97 ∗           | 24.33 ∗∗∗       |

Note:

∗ p &lt; 0.05; ∗∗ p &lt; 0.01; ∗∗∗ p &lt; 0.001

Supplementary Table 4: Linear model with robust standard errors clustered at the participant and headline levels predicting change in belief rating between pre- and post AI explanation feedback to compare effects of deceptive and honest explanation feedback for true and false news headlines. We use a deceptive classifications dummy variable (0=honest, 1=deceptive) and limit the analysis to AI explanations and news headlines observations only.

## Dependent variable: Belief Rating

|                                                        | News Headlines   | Trivia Items     |
|--------------------------------------------------------|------------------|------------------|
| Constant                                               | 1.66 ∗∗∗ (0.10)  | 2.28 ∗∗∗ (0.15)  |
| True Statement                                         | 2.04 ∗∗∗         | 1.55 ∗∗∗         |
|                                                        | (0.13)           | (0.20)           |
| Deception                                              | 0.92 ∗∗∗         | 1.20 ∗∗∗         |
|                                                        | (0.09)           | (0.12)           |
| Logically Invalid                                      | 0.12             | 0.09             |
| Deception * True Statement                             | (0.09) -2.14 ∗∗∗ | (0.15) -2.69 ∗∗∗ |
|                                                        | (0.12)           | (0.16)           |
| True * Logically Invalid Deception * Logically Invalid | -0.10 (0.12)     | 0.01 (0.20)      |
|                                                        | -0.35 ∗          | -0.13            |
|                                                        | (0.14)           | (0.19)           |
| True * Deception * Logically Invalid                   | 0.31             | 0.18             |
|                                                        | (0.20)           | (0.31)           |
| Observations                                           | 11,780           | 12,200           |
| Participants                                           | 589              | 610              |
| R 2                                                    | 0.30             | 0.23             |
| 2                                                      |                  |                  |
| Adjusted R                                             | 0.30             | 0.23             |
| Residual Std. Error                                    | 1.13             | 1.26             |
| F Statistic                                            | 103.59 ∗∗∗       | 85.10 ∗∗∗        |

Note:

∗ p &lt; 0.05; ∗∗ p &lt; 0.01; ∗∗∗ p &lt; 0.001

Supplementary Table 5: Linear model with robust standard errors clustered at the participant and headline levels predicting belief rating across logical validity of explanations, the Explanations, and deceptive AI feedback. We use a logical invalid dummy variable (0=logically valid, 1=logically invalid), a veracity dummy variable (0=false, 1=true), a deceptive classifications dummy variable (0=honest, 1=deceptive), and an explanations dummy variable indicating whether the participant a classification with or without explanation (0=No explanation, 1=Explanation).

## Dependent variable: Belief Rating

|                                                                    | News Headlines            | Trivia Items           |
|--------------------------------------------------------------------|---------------------------|------------------------|
| Constant                                                           | 1.72 ∗∗∗ (0.11)           | 2.28 ∗∗∗ (0.16)        |
| True Statement                                                     | 1.88 ∗∗∗                  | 1.49 ∗∗∗               |
| Deception                                                          | (0.14) 0.70 ∗∗∗           | (0.20) 1.20 ∗∗∗ (0.10) |
| Explanation                                                        | (0.09) -0.03              | -0.02 (0.06)           |
| Knowledge                                                          | (0.06) -0.11 ∗∗           | -0.11                  |
|                                                                    | (0.04) -1.63 ∗∗∗          | (0.07) -2.52 ∗∗∗       |
| Deception * True Statement                                         | (0.13)                    | (0.16)                 |
|                                                                    |                           | 0.14                   |
| Explanation * True Statement                                       | 0.18 ∗ (0.09)             | (0.11)                 |
| Deception * Explanation                                            | 0.26 ∗ (0.12)             | -0.00 (0.14)           |
| True * Explanation * Deception True * Knowledge                    | -0.67 ∗∗∗ (0.18) 0.44 ∗∗∗ | -0.27 (0.22) 0.35 ∗∗   |
| Explanation * Knowledge                                            | (0.07) -0.07              | (0.12) 0.06            |
|                                                                    | (0.05)                    | (0.07)                 |
| Deception * Knowledge                                              | -0.05 (0.09)              | 0.06 (0.05)            |
| True * Explanation * Knowledge Explanation * Deception * Knowledge | -0.01                     | -0.05                  |
|                                                                    | (0.07) 0.18 ∗             | (0.10) -0.16 (0.10)    |
| True * Deception * Knowledge                                       | (0.08) 0.02               | -0.08                  |
|                                                                    | (0.14)                    | (0.11)                 |
| True * Explanation * Deception *                                   | -0.19                     | 0.16                   |
| Knowledge                                                          |                           |                        |
|                                                                    | (0.15)                    | (0.16)                 |
| Observations Participants                                          |                           |                        |
|                                                                    | 11,780                    | 12,200                 |
| 2                                                                  | 589                       | 610                    |
| R                                                                  | 0.33                      | 0.24                   |
| Adjusted R 2                                                       | 0.33                      |                        |
|                                                                    |                           | 0.24                   |
| Residual Std. Error                                                | 1.11 90.70                | 1.25                   |
|                                                                    |                           | 76.12                  |
| F Statistic                                                        | ∗∗∗                       | ∗∗∗                    |

Note:

∗ p &lt; 0.05; ∗∗ p &lt; 0.01; ∗∗∗ p &lt; 0.001

Supplementary Table 6: Linear model with robust standard errors clustered at the participant and headline levels predicting belief rating across participants' self-reported prior knowledge ratings (z-scored), explanation and no explanation AI feedback. We use a prior knowledge variable, a veracity dummy variable (0=false, 1=true), a deceptive classifications dummy variable (0=honest, 1=deceptive), and an explanations dummy variable indicating whether the participant a classification with or without explanation (0=No explanation, 1=Explanation).

## Dependent variable: Belief Rating

|                                        | News Headlines   | Trivia Items    |
|----------------------------------------|------------------|-----------------|
| Constant                               | 1.68 ∗∗∗ (0.11)  | 2.30 ∗∗∗ (0.15) |
| True Statement                         | 1.96 ∗∗∗         | 1.47 ∗∗∗        |
|                                        | (0.14)           | (0.22)          |
| Deception                              | 0.81 ∗∗∗         | 1.14 ∗∗∗        |
|                                        | (0.10)           | (0.09)          |
| Explanation                            | -0.07            | -0.03           |
|                                        | (0.06)           | (0.06)          |
| Trust                                  | -0.03            | -0.12 ∗∗        |
|                                        | (0.03)           | (0.05)          |
| Deception * True Statement             | -1.87 ∗∗∗        | -2.47 ∗∗∗       |
|                                        | (0.14)           | (0.14)          |
| Explanation * True Statement           | 0.20 ∗           | 0.11            |
|                                        | (0.09)           | (0.11)          |
| Deception * Explanation                | 0.25 ∗           | 0.03            |
|                                        | (0.13)           | (0.15)          |
| True * Explanation * Deception         | -0.61 ∗∗         | -0.31           |
|                                        | (0.19)           | (0.21)          |
| True * Trust                           | 0.13 ∗           | 0.35 ∗∗∗        |
|                                        | (0.05)           | (0.08)          |
| Explanation * Trust                    | 0.04             | 0.05            |
|                                        | (0.06)           | (0.05)          |
| Deception * Trust                      | 0.35 ∗∗∗         | 0.38 ∗∗∗        |
|                                        | (0.05)           | (0.09)          |
| True * Explanation * Trust             | 0.01             | -0.04           |
|                                        | (0.09)           | (0.09)          |
| Explanation * Deception * Trust        | -0.01            | -0.05           |
|                                        | (0.09)           | (0.10)          |
| True * Deception * Trust               | -0.58 ∗∗∗        | -0.84 ∗∗∗       |
|                                        | (0.10)           | (0.14)          |
| True * Explanation * Deception * Trust | -0.11            | 0.10            |
|                                        | (0.15)           | (0.19)          |
| Observations                           | 11,780           | 12,200          |
| Participants                           | 589              | 610             |
| R 2                                    | 0.32             | 0.24            |
| Adjusted R 2                           | 0.32             | 0.24            |
| Residual Std. Error                    | 1.11             | 1.25            |
| F Statistic                            | 77.93 ∗∗∗        | 129.30 ∗∗∗      |
|                                        | ∗ ∗∗             | ∗∗∗             |

Note:

∗ p &lt; 0.05; ∗∗ p &lt; 0.01; ∗∗∗ p &lt; 0.001

Supplementary Table 7: Linear model with robust standard errors clustered at the participant and headline levels predicting belief rating across participants' trust in AI systems rating (z-scored), and explanation and no explanation AI feedback. We use a trust variable, a veracity dummy variable (0=false, 1=true), a deceptive classifications dummy variable (0=honest, 1=deceptive), and an explanations dummy variable indicating whether the participant a classification with or without explanation (0=No explanation, 1=Explanation).

## Dependent variable: Belief Rating

|                                | News Headlines   | Trivia Items    |
|--------------------------------|------------------|-----------------|
| Constant                       | 1.68 ∗∗∗ (0.11)  | 2.29 ∗∗∗ (0.15) |
| True Statement                 | 1.94 ∗∗∗         | 1.52 ∗∗∗        |
|                                | (0.14)           | (0.22)          |
| Deception                      | 0.71 ∗∗∗         | 1.19 ∗∗∗        |
|                                | (0.10)           | (0.10)          |
| Explanation                    | -0.06            | -0.02           |
|                                | (0.06)           | (0.06)          |
| CRT                            | -0.06            | -0.08 ∗         |
|                                | (0.03)           | (0.04)          |
| Deception * True Statement     | -1.72 ∗∗∗        | -2.57 ∗∗∗       |
|                                | (0.14)           | (0.15)          |
| Explanation * True Statement   | 0.22 ∗           | 0.11            |
|                                | (0.08)           | (0.11)          |
| Deception * Explanation        | 0.32 ∗∗          | 0.01            |
|                                | (0.12)           | (0.15)          |
| True * Explanation * Deception | -0.72 ∗∗∗        | -0.28           |
|                                | (0.19)           | (0.22)          |
| True * CRT                     | 0.01             | 0.13            |
|                                | (0.06)           | (0.08)          |
| Explanation * CRT              | -0.02            | -0.03           |
|                                | (0.03)           | (0.06)          |
| Deception * CRT                | -0.10            | 0.05            |
|                                | (0.07)           | (0.07)          |
| True * Explanation * CRT       | 0.13             | -0.06           |
|                                | (0.08)           | (0.10)          |
| Explanation * Deception * CRT  | 0.13             | -0.07           |
|                                | (0.09)           | (0.09)          |
| True * Deception * CRT         | 0.17             | -0.14           |
|                                | (0.12)           | (0.14)          |
| * CRT                          | -0.25            | 0.19            |
| True * Explanation * Deception | (0.17)           | (0.18)          |
| Observations                   | 11,780           | 12,200          |
| Participants                   | 589              | 610             |
| R 2                            | 0.30             | 0.23            |
| Adjusted R 2                   | 0.30             | 0.23            |
| Residual Std. Error            | 1.13             | 1.26            |
| F Statistic                    |                  | ∗∗∗             |
|                                | 65.68 ∗∗∗        | 93.91           |

Note:

∗ p &lt; 0.05; ∗∗ p &lt; 0.01; ∗∗∗ p &lt; 0.001

Supplementary Table 8: Linear model with robust standard errors clustered at the participant and headline levels predicting belief rating across z-scored cognitive reflection test (CRT) score, explanation and no explanation AI feedback. We use a CRT variable, a veracity dummy variable (0=false, 1=true), a deceptive classifications dummy variable (0=honest, 1=deceptive), and an explanations dummy variable indicating whether the participant a classification with or without explanation (0=No explanation, 1=Explanation).

## Dependent variable: Prior Knowledge Rating

|                     | False            | True         |
|---------------------|------------------|--------------|
| Constant            | 0.30 ∗∗∗ (0.08)  | -0.07 (0.08) |
| Trivia              | -0.38 ∗∗∗ (0.11) | 0.01 (0.15)  |
| Observations        | 11,705           | 12,275       |
| R 2                 | 0.03             | 0.00         |
| Adjusted R 2        | 0.03             | -0.00        |
| Residual Std. Error | 1.00             | 0.98         |
| F Statistic         | 12.91 ∗∗         | 0.00         |

Note:

∗ p &lt; 0.05; ∗∗ p &lt; 0.01; ∗∗∗ p &lt; 0.001

Supplementary Table 9: Linear model with robust standard errors clustered at the participant and headline levels predicting z-scored self-reported prior knowledge for true and false news and trivia items. We use a trivia items dummy variable (0=news headlines, 1=trivia items).

## Dependent variable: Belief Rating

|                                           | News Headlines     | Trivia Items        |
|-------------------------------------------|--------------------|---------------------|
| Constant                                  | 1.01 ∗ (0.39)      | -0.26 (0.44)        |
| True                                      | -1.81 ∗∗ (0.70)    | 0.43 (0.62)         |
| Number of Premises                        | -0.13 ∗            | 0.24                |
| Number of Premises * True                 | (0.06)             | (0.14)              |
|                                           | 0.10 (0.09)        | -0.54 ∗∗ (0.17)     |
| Perceived Truthfulness of Premises        | -0.79              | -0.15               |
| Perceived Truthfulness of Premises * True | (0.48) 0.81 (0.79) | (0.38) -0.30 (0.57) |
| Perceived Logical Validity                | 1.26 ∗∗∗ (0.19)    | 0.73 ∗∗ (0.28)      |
| Perceived Logical Validity * True         | -1.36 ∗∗ (0.51)    | -1.03 ∗ (0.46)      |
| Observations                              | 3023               | 3129                |
| R 2                                       | 0.32               | 0.28                |
| Adjusted R 2                              | 0.32               | 0.28                |
| Residual Std. Error                       | 1.11               | 1.26                |
|                                           |                    | ∗∗∗                 |
| F Statistic                               | 34.36 ∗∗∗          | 34.48               |

Note:

∗ p &lt; 0.05; ∗∗ p &lt; 0.01; ∗∗∗ p &lt; 0.001

Supplementary Table 10: Linear model with robust standard errors clustered at the participant and headline levels predicting the change in belief after explanation for true and false news headlines and trivia items. We use number of premises within an explanation, perceived truthfulness of premises within an explanation, perceived logical validity of an explanation and its classification, and their interaction with a dummy of the headline being true or false (true = 1, false = 0) as predictors.

## Dependent variable: Belief Rating

|                     | News Headlines   | Trivia Items   |
|---------------------|------------------|----------------|
| Constant            | 0.36 (0.42)      | -0.27 (0.54)   |
| Word Count          | 0.01             | 0.03 ∗∗        |
|                     | (0.01)           | (0.01)         |
| Word Count * True   | -0.03 ∗∗         | -0.05 ∗∗       |
| Reading Ease        | (0.01)           | (0.02) 0.00    |
|                     | 0.00 (0.01)      | (0.00)         |
| Reading Ease * True | -0.01 ∗          | -0.01 ∗        |
|                     | (0.01)           | (0.00)         |
| Observations        | 3023             | 3129           |
| R 2                 | 0.30             | 0.26           |
| Adjusted R 2        | 0.30             | 0.26           |
| Residual Std. Error | 1.13             | 1.28           |
| F Statistic         | 34.82 ∗∗∗        | 36.46 ∗∗∗      |

Note:

∗ p &lt; 0.05; ∗∗ p &lt; 0.01; ∗∗∗ p &lt; 0.001

Supplementary Table 11: Linear model with robust standard errors clustered at the participant and headline levels predicting the change in belief after deceptive explanations for true and false news headlines and trivia items. We use the word count and reading ease of an explanation, and their interaction with a dummy of the news headline being true or false (true = 1, false = 0) as predictors.
Valdemar Danry Pat Pataranutaporn Matthew Groh Ziv Epstein Pattie Maes
Deceptive AI systems that give explanations are more convincing than honest AI systems and can amplify belief in misinformation
Advanced Artificial Intelligence (AI) systems, specifically large language
models (LLMs), have the capability to generate not just misinformation, but
also deceptive explanations that can justify and propagate false information
and erode trust in the truth. We examined the impact of deceptive AI generated
explanations on individuals' beliefs in a pre-registered online experiment with
23,840 observations from 1,192 participants. We found that in addition to being
more persuasive than accurate and honest explanations, AI-generated deceptive
explanations can significantly amplify belief in false news headlines and
undermine true ones as compared to AI systems that simply classify the headline
incorrectly as being true/false. Moreover, our results show that personal
factors such as cognitive reflection and trust in AI do not necessarily protect
individuals from these effects caused by deceptive AI generated explanations.
Instead, our results show that the logical validity of AI generated deceptive
explanations, that is whether the explanation has a causal effect on the
truthfulness of the AI's classification, plays a critical role in countering
their persuasiveness - with logically invalid explanations being deemed less
credible. This underscores the importance of teaching logical reasoning and
critical thinking skills to identify logically invalid arguments, fostering
greater resilience against advanced AI-driven misinformation.