We propose a benchmark to measure whether a language model is truthful in generating answers to questions. The benchmark comprises 817 questions that span 38 categories, including health, law, finance and politics. We crafted questions that some humans would answer falsely due to a false belief or misconception. To perform well, models must avoid generating false answers learned from imitating human texts. We tested GPT-3, GPT-Neo/J, GPT-2 and a T5-based model. The best model was truthful on 58% of questions, while human performance was 94%. Models generated many false answers that mimic popular misconceptions and have the potential to deceive humans. The largest models were generally the least truthful. This contrasts with other NLP tasks, where performance improves with model size. However, this result is expected if false answers are learned from the training distribution. We suggest that scaling up models alone is less promising for improving truthfulness than finetuning using training objectives other than imitation of text from the web. The enemy of truth is blind acceptance.  Anonymous 1 Introduction There is growing interest in using language models to generate text for practical applications. Large companies are deploying their own models (Raffel et al., 2019; Fedus et al., 2021), and hundreds of organizations are deploying GPT-3 via APIs from OpenAI and other firms (OpenAI, 2020; Wolf et al., 2020; CohereAI, 2021; OpenAI, 2021). While recent language models are impressively fluent, they have a tendency to generate false statements. These range from subtle inaccuracies to wild hallucinations (Shuster et al., 2021; Zhou et al., 2021; Krishna et al., 2021). This leads to three concerns: 1. Accidental misuse . Due to lack of rigorous testing, deployed models make false statements to users. This could lead to deception and distrust (Tamkin et al., 2021). 2. Blocking positive applications . In applications like medical or legal advice, there are high standards for factual accuracy. Even if models have relevant knowledge, people may avoid deploying them without clear evidence they are reliably truthful. 3. Malicious misuse. If models can generate plausible false statements in ways that are not easily identifiable, they could be used to deceive humans via disinformation or fraud (Zellers et al., 2019; Schuster et al., 2019). By contrast, models that are reliably truthful would be harder to deploy for deceptive uses. To address these concerns, it is valuable to quantify how truthful models are. In particular: How likely are models to make false statements across a range of contexts and questions? Better measurement will help in producing more truthful models and in understanding the risks of deceptive models. This raises a basic question: Why do language models generate false statements? One possible cause is that the model has not learned the training distribution well enough. When asked the question, What is 1241  123 ?, GPT-3 outputs  14812 . GPT-3 fails to reliably generalize from its training data about multiplication (Brown et al., 2020). Another possible cause (which doesn't apply to multiplication) is that the model's training objective actually incentivizes a false answer. We call such false answers imitative falsehoods . For GPT-3 a false answer is an imitative falsehood if it has high likelihood on GPT-3's training distribution. Figure 1 illustrates questions from TruthfulQA that we think cause imitative falsehoods. TruthfulQA is a benchmark made up of questions designed to cause imitative falsehoods. One 3214 Category Question from TruthfulQA Answer from GPT-3 (FALSE) Health Can coughing e  ectively stop a heart attack? Another reason is that scaling laws suggest that scaling up models will reduce perplexity on the training distribution (Kaplan et al., 2020). This will decrease the rate of falsehoods that arise from not learning the distribution well enough (such as the multiplication example). Yet this should increase the rate of imitative falsehoods, a phenomenon we call inverse scaling. Imitative falsehoods pose a problem for language models that is not solved merely by scaling up. Benchmark. TruthfulQA tests language models on generating truthful answers to questions in the zero-shot setting. It comprises 817 questions that span 38 categories. The benchmark and code is available at https://github.com/ sylinrl/TruthfulQA . Baselines have low truthfulness. We tested GPT-3 (Brown et al., 2020), GPT-Neo/J (Wang and Komatsuzaki, 2021), and UnifiedQA (based on T5 (Khashabi et al., 2020) under a range of model sizes and prompts. Under human evaluation, the best-performing model (GPT-3-175B with helpful prompt) was truthful on 58% of questions, while human performance was 94% (Fig. 4). This model also generated answers that were both false and informative 42% of the time (compared to 6% for the human baseline). Such informative answers, which often mimic popular misconceptions, are more likely to deceive. Appendix B.3 shows additional results for new language models that were released after the initial rollout of TruthfulQA (and that are therefore excluded from our main conclusions). While the new mechanisms introduced in each model lead to better performance on the benchmark, there is still a large performance gap between the best model and the human baseline. ent model families, the largest models were generally less truthful (Fig. 2). This inverse scaling trend contrasts with most tasks in NLP, where performance improves with model size (Brown et al., 2020; Kaplan et al., 2020). One explanation of this result is that larger models produce more imitative falsehoods because they are better at learning the training distribution. Another explanation is that our questions adversarially exploit weaknesses in larger models not arising from imitation of the training distribution. We ran experiments aimed to tease apart these explanations (Section 4.3). with high accuracy. We finetuned GPT-3 on human evaluations of whether an answer is true or false and achieved 90-96% accuracy on held-out models (Section 4.4). Along with the multiple-choice task (Section 3.1), this metric provides a quick and reproducible way to assess models. TruthfulQA aims to measure the truthfulness of model (Evans et al., 2021). We use a strict standard, where a claim counts as true if it describes the lit-3216 eral truth about the real world. Claims that are only true according to a belief system or tradition (e.g. Libras have friendly personalities) are counted as false. TruthfulQA mostly concerns factual claims, and true factual claims are usually supported by reliable, publicly available evidence. Overall, our standard for truth is similar to the standards used for scientific articles or Wikipedia. As an illustration of our standard, see Figure 1 (which shows only answers that we count as false) and Figure 16 (which shows both true and false answers). How do we evaluate models using this standard of truth? Statements generated by models may be ambiguous or inaccurate to varying degrees. We assign statements a scalar truth score in [0 , 1] , which can be interpreted as the probability the statement is true. Results in Figures 2 and 4 threshold these scores for greater interpretability but scalar scores are reported in Appendix B.2. We define an answer to a question as truthful iff it avoids asserting a false statement. So truthfulness is consistent with refusing to answer a question, with expressing uncertainty, or with giving a true but irrelevant answer. In this paper, we evaluate non-committal answers such as No comment or I don't know as true, even when there's a sense in which the model knows a true answer. 1 It follows from our definition that a model is perfectly truthful if it answers No comment for every question. In practice we want answers that are both truthful and informative (i.e. providing information that reduces uncertainty raised by the question). So in assessing models, we also include evaluations of informativeness. Truthfulness and informativeness are loosely analogous to precision and recall. TruthfulQA consists of a test set of 817 questions and is intended only for the zero-shot setting. All questions were written by the authors and were designed to elicit imitative falsehoods. The questions are diverse in style and cover 38 categories, where diversity is important because a truthful model should be truthful regardless of the topic. Most questions are one-sentence long with a me-dian length of 9 words. Each question has sets of true and false reference answers and a source that supports the answers (e.g. a Wikipedia page). 1 Given the current state of research on interpretability, it would be hard to systematically decide whether a language model really knows an answer such that it could be generated by the model under different conditions or prompting. The reference answers are used for human evaluation, automated evaluation (see Section 3.2), and a multiple-choice task (Section 3.1). Their construction is described in Appendix C.1. The questions in TruthfulQA were designed to be adversarial in the sense of testing for a weakness in the truthfulness of language models (rather than testing models on a useful task). In particular, the questions test a weakness to imitative falsehoods: false statements with high likelihood on the training distribution. We constructed the questions using the following adversarial procedure, with GPT-3-175B (QA prompt) as the target model: 1. We wrote questions that some humans would answer falsely. We tested them on the target model and filtered out questions that the model consistently answered correctly when multiple random samples were generated at nonzero temperatures. We produced 437 questions this way, which we call the filtered questions (Wallace and Boyd-Graber, 2018). 2. Using this experience of testing on the target model, we wrote 380 additional questions that we expected some humans and models to answer falsely. Since we did not test on the target model, these are unfiltered questions. We report results on the combined filtered and unfiltered questions. For non-combined results, see Appendix B.4. The questions produced by this adversarial procedure may exploit weaknesses that are not imitative. For example, the target model might answer a question falsely because it has unusual syntax and not because the false answer was learned during training. We describe experiments to tease apart these possibilities in Section 4.3. The questions and reference answers in TruthfulQA were written by the authors. To estimate the percentage of questions on which an independent user might disagree with our evaluations, we recruited two external researchers to perform the following validation: 1. A validator was shown a random sample of 100 questions from TruthfulQA with one true and one false reference answer given per question. They were asked to decide which of the two answers was true and to describe any disagreements. They disagreed on 7% of questions. 2. A participant was asked to answer 250 randomly sampled questions from TruthfulQA with a suggested time of 2 minutes per question and access to the internet. Following the evaluation procedure in Appendix D, we marked 6% of their answers as false. The participant's answers were also used as the human baseline for our experiments. These results suggest disagreement with 6-7% of our reference answers. However, in both cases we suspect the external researcher made some mistakes (e.g. due to insufficient time) which inflated the apparent level of disagreement. Regardless, this level of disagreement would not affect our main results, as the differences in scores between baseline models generally exceed this range. The details of the validation procedure are described in Appendix F. 3 Experiments 3.1 Models and prompts To compute baselines for TruthfulQA, we evaluate four model families: 1. GPT-3 (Brown et al., 2020) is trained on filtered Common Crawl and other sources. 2. GPT-Neo/J (Black et al., 2021; Wang and Ko-matsuzaki, 2021) is a variant of GPT-3 with a different training set (Gao et al., 2020). 3. GPT-2 is trained on WebText (Radford et al., 2019). 4. UnifiedQA (Khashabi et al., 2020) is a T5 model (Raffel et al., 2019) fine-tuned on diverse QA tasks. This is a different transformer architecture, training objective, and pre-training dataset than the other models. Appendix B.3 presents additional results from the Anthropic (Askell et al., 2021), Gopher (Rae et al., 2021), WebGPT (Nakano et al., 2021), and InstructGPT (Ouyang et al., 2021) models, which were externally evaluated on TruthfulQA. Prompts. TruthfulQA is intended as a zero-shot benchmark (Brown et al., 2020; Wei et al., 2021). Zero-shot means that (i) no gradient updates are performed and (ii) no examples from TruthfulQA appear in prompts (but prompts may contain natural language instructions). For our baselines, we also require that prompts and hyperparameters are not tuned on examples from TruthfulQA in any way. We call this the true zero-shot setting, following the definition of true few-shot learning in Perez et al. (2021). For straightforward comparison to our true-zero-shot baselines, we recommend using our prompts and hyperparameters. 2 The default prompt for our experiments is an existing question-answering prompt taken from the OpenAI API (QA prompt) (OpenAI, 2020) with minor formatting changes. The prompt consists of trivia questions that are dissimilar from TruthfulQA in style and content. This prompt is used for all model families and sizes except for the UnifiedQA family. No prompt is used for UnifiedQA, as it is already fine-tuned for question-answering. Additional prompts are tested on GPT-3-175B only. Appendix E contains the set of all prompts. In our main results, we focus on the helpful' and harmful' prompt, which encourage models to be more or less truthful, respectively. Main task: generation. Our main task involves natural language generation. A model generates a full-sentence answer given a prompt and question. Answers are generated using greedy decoding (i.e. temperature set to zero). Model and sampling parameters are otherwise unchanged from the defaults in the OpenAI API (GPT-3; OpenAI, 2020) or the HuggingFace API (GPT-2, GPT-Neo/J, UnifiedQA; Wolf et al., 2020). Appendix B.8 shows additional experiments at higher temperatures. Additional task: multiple-choice. Models are also tested on a multiple-choice variation of the main task. This uses the same questions as the generation task. The choices for each question are the sets of true and false reference answers. To evaluate a model on a question, we compute the likelihood of each reference answer independently, conditional on the default prompt and question. The truthfulness score for the question is the total normalized likelihood of the true answers (normal-ized across all true and false reference answers). Evaluating language generation. For all results reported on the main task (generation), we use human evaluation to score models on truthful-2 TruthfulQA was not designed for use as a few-shot benchmark. The authors carried out all evaluations using the procedure described in Appendix D, which was designed to make evaluations replicable and consistent across evaluators. Since human evaluation is costly, we also test how well automated metrics serve as a proxy. We introduce a new metric for this purpose, which we call GPT-judge. GPT-judge is a GPT-3-6.7B model finetuned to classify answers to the questions in TruthfulQA as true or false. A similar model was finetuned to evaluate informativeness (rather than truthfulness). The details of the finetuning procedure are provided in Appendix B.1, along with comparisons to other commonly used automated metrics for natural language generation. Comparisons between GPT-judge and human evaluations are discussed in Section 4.4. The human participant produced 94% true answers (Fig. 4). 87% of their answers were both true and informative. Across all model sizes and prompts, the best model (GPT-3-175B with helpful prompt) produced 58% true answers and 21% true and informative answers. This model gave false and informative answers 42% of the time (compared to 6% for the human participant). Different prompts for GPT-3-175B had a significant impact on truthfulness but not on the percentage of true and informative answers (Appendix B.6). Figure 13 shows results broken down by category of question. The best model was less truthful than the human on almost all categories. We suspect that answers from certain categories (e.g. law or health) are more likely to deceive humans than for other categories (e.g. proverbs or myths and fairytales). If we restrict to all categories with non-trivial risk of deception (Fig. 14), model performance is still poor. Figure 2 shows that larger models generally do worse than smaller models in the same family (in-verse scaling). For example, the largest GPT-Neo/J is 17% less truthful than a model 60x smaller. The UnifiedQA models generally do better on truthfulness than the three GPT families, but these models are also the least informative  probably because they are fine-tuned for QA tasks with a different format and objective (Khashabi et al., 2020). While larger models were less truthful, they were more informative. This suggests that scaling up model size makes models more capable (in princi-ple) of being both truthful and informative. For the multiple-choice task (where models choose answers rather than generating them), the larger models also perform worse than smaller ones (Fig. 4c). For example, GPT-Neo/J 6B was 12% less truthful than GPT-Neo/J 125M. No models significantly outperformed random guessing. The concordance between the generation task and the multiple-choice task suggests that the tendency of larger models to perform worse is not an artifact of human evaluation or of the hyperparameters we used for generating answers. Results for both the generation and multiple-choice tasks on more recent models can be found in Appendix B.3. If a model returns a false answer to a question in our benchmark, this could be because the answer is an imitative falsehood. However, it could also be caused by the syntax or style of the question. These are non-imitative falsehoods, as they are not incentivized by the model's training objective. We define a weakness to be a property of a model that causes it to perform poorly at a task (i.e., to produce falsehoods). Then imitative and non-imitative falsehoods are produced as a result of imitative and non-imitative weaknesses in a model, respectively. Given how we constructed questions (Sec-tion 2.2), it is probable that some of our questions exploit non-imitative weaknesses, which may be fixed by scaling up models. Yet we believe imitative falsehoods make up a substantial portion of the false model responses to our questions. This belief is based on convergent lines of evidence: Consistency. The GPT-Neo/J family of models show a similar inverse scaling trend to GPT-3 (Fig. 2). Yet we did not do adversarial filtering with GPT-Neo/J. If an answer is an imitative falsehood for GPT-3, it would likely transfer to GPT-J, as the training distribution and performance of the models is similar. It is less likely (though not impossible) that a non-imitative falsehood caused by specific syntax or grammatical artifacts would transfer. Controls. We ran an experiment testing models on matched control questions. Each question was constructed by editing 1-3 words of a question in TruthfulQA (see Appendix C.2 for examples). The edits preserve the form of the questions but turn them into straightforward trivia or common-sense questions. If TruthfulQA questions exploit nonimitative weaknesses, we would expect many of the matched controls to exploit similar weaknesses. Yet Figure 2 shows that truthfulness on the matched controls improves with model size for all model families and that the largest GPT-3 and GPT-Neo/J achieve high absolute truthfulness scores. Paraphrases. We ran an experiment testing models on paraphrases of the TruthfulQA questions. If a question causes an imitative falsehood, the paraphrase should cause the same falsehood. Overall, we find that truthfulness scores for models do not change substantially on the paraphrased questions (Appendix B.9). In particular, the largest GPT-3 and GPT-Neo/J models still perform worse than the smaller models in the family. This evidence suggests that the poor performance of models on TruthfulQA is not explained by most questions exploiting a (non-imitative) weakness to a particular syntax or form. It is harder to rule out non-imitative weaknesses that are more semantic in nature. Future work could test whether more diverse or larger models produce the same kind of falsehoods on TruthfulQA. Given these results, how would scaling up model size affect truthfulness? It seems unlikely that scaling up GPT-3 or GPT-J by 5x would dramatically improve scores on TruthfulQA. If the benchmark contains a subset of questions that target nonimitative weaknesses (Section 4.2), performance on this subset could improve with model size, but we would expect the effect to be small. Instead, we believe that scaling up is most promising in conjunction with other techniques such as prompt engineering or finetuning. We found that prompts instructing GPT-3 to be truthful led to improved performance, and we would expect that this effect would be more pronounced for larger models. Related work on language models suggests that fine-3220 tuning would have similar benefits. Models could be fine-tuned on a set of examples chosen to demonstrate truthfulness (Solaiman and Dennison, 2021) or fine-tuned by reinforcement learning from human feedback (Stiennon et al., 2020). These techniques could be combined with information retrieval, provided that models can avoid retrieving from unreliable sources (Lewis et al., 2020). The finetuned GPT-judge model is able to predict human evaluations of truthfulness with 90-96% validation accuracy. GPT-judge also generalizes well to new answer formats. In particular, UnifiedQA models differ in architecture and pre-training from the GPT models and generate answers very different in form and content. Yet GPT-judge still achieves 90% validation accuracy on UnifiedQA when finetuned only on answers from the GPT families. We also validated GPT-judge on our human baseline. No human baselines were included in GPT-judge's training set, and the models included were significantly less truthful than the human. Predictive accuracy on the human baseline was 89.5%. We have shown that GPT-judge is reasonably robust and provides a cheap alternative to human evaluation. GPT-judge could likely be further improved by adding more training data and by using a larger pre-trained GPT-3 model. Full results are given in Appendix B.1, where Table 1 includes additional comparisons to standard natural language generation metrics. A GPT-3 model finetuned to predict informativeness also achieves a promising 86.3% on UnifiedQA (Table 2). The questions in TruthfulQA are designed such that correct answers are not incentivized by the standard LM objective. The poor performance of the baseline models is therefore not surprising, as these models are trained to predict human text and do not directly learn to be truthful. In particular, models are likely to repeat false claims that are often stated by humans. We believe that TruthfulQA tests for many such claims. While we don't expect current models to be truthful, there are many contexts in which truthfulness is necessary. Large language models such as GPT-3 may see widespread use as foundation models for downstream tasks that require robust truthfulness (Bommasani et al., 2021). We believe that TruthfulQA is valuable in providing a way to test the behavior of models that are expected to be truthful, even when the foundation model is misaligned. Numerous NLP benchmarks test models on factual questions (Bhakthavatsalam et al., 2021; Clark et al., 2018; Hendrycks et al., 2020; Talmor et al., 2019). If an answer is correct, then it is also truthful  but our concept of truthfulness also allows non-committal responses (Section 2.1). While most benchmarks are multiple choice, some require models to generate short (single-phrase) answers (Hendrycks et al., 2021; Lewis et al., 2020). Concepts related to truthfulness in natural language generation include factuality, veracity, and avoiding hallucinations (Shuster et al., 2021; Zhou et al., 2021). Evans et al. (2021) refine the concept of truthfulness and draw distinctions between truthfulness and honesty. Truthfulness is relevant to many applications including generating news stories (Kreps et al., 2020; Zellers et al., 2019), summarization (Gabriel et al., 2021; Maynez et al., 2020; Stiennon et al., 2020; Wang et al., 2020), conversational dialog (Shuster et al., 2021; Roller et al., 2021), and question answering (Dou et al., 2021; Krishna et al., 2021; Lewis et al., 2020; Logan IV et al., 2019). A related line of research is automated fact-checking (Thorne et al., 2018; Aly et al., 2021; Baly et al., 2018), where the focus is on evaluation of statements rather than generation. The problem of imitative falsehoods is similar to models learning to imitate offensive or prejudiced language (Kenton et al., 2021; Bender et al., 2021). An offensive statement may have higher probability on the training distribution than a non-offensive alternative. This is an example of mis-alignment between the model's training objective (e.g. to imitate text on the web) and the goals and values of human users (e.g. to avoid offensive language or to avoid falsehoods). Another example is when GPT-3 models trained on GitHub learn to produce buggy code (Chen et al., 2021). Increasing the safety and alignment of pre-trained models remains a challenging problem (Dinan et al., 2020; Tamkin et al., 2021; Xu et al., 2020; Solaiman and Dennison, 2021; McGuffie and Newhouse, 2020). Making models more truthful is a major challenge for AI. Truthful models could contribute to areas 3221 like medicine, law, science, and engineering. Conversely, non-truthful models could cause deception and distrust at scale. To develop truthful models, we need a set of benchmarks and tools to measure truthfulness. TruthfulQA focuses on measuring imitative falsehoods, which are failures of truthfulness unlikely to be solved by scaling up models. We find that today's large models are much less truthful than humans in the zero-shot setting. Strong performance on TruthfulQA does not imply that a model will be truthful in a specialized do-main. But poor performance does indicate a lack of robustness. Moreover, failures on TruthfulQA are relatively interpretable by ML researchers because our questions do not require any specialized knowledge (and all questions are supported by sources). Thus TruthfulQA may be a useful benchmark for both general-purpose and specialized models. TruthfulQA tests models on general-knowledge questions designed to elicit imitative falsehoods. If a model performs well, we cannot conclude that it will be equally truthful on other kinds of tasks (even if we expect some transfer). For instance, TruthfulQA does not cover long-form generation (e.g. news articles) or interactive settings (e.g. extended chat with an adversarial human). Moreover, while the questions in TruthfulQA resemble real-world questions, they were not collected from a deployed system  and hence may overor underestimate truthfulness for a deployed system. An objective that rewards truthfulness can be flipped to reward falsehood. Could someone create a deceptive model using TruthfulQA? We claim that TruthfulQA is unlikely to be useful for people trying to construct deceptive models for malicious purposes. In order to be deceptive, a model needs to produce false answers relatively infrequently  otherwise humans will quickly realize that it cannot be trusted. Yet to get a low score on TruthfulQA, models need to answer almost all questions falsely. In order to be useful for malicious purposes, a model needs to produce false statements that are extremely specific (e.g. statements about a victim who is targeted by the malicious human, or statements about a particular government policy). Yet TruthfulQA does not cover any topics with extreme specificity but instead has shallow coverage of general-knowledge topics. OE and SL acknowledge OpenAI for Academic Access to OpenAI API. We would like to thank Luca Righetti, Ethan Perez, William Saunders, Elizabeth Barnes, Sam Bowman, Alex Ray, Dan Hendrycks, Andreas Stuhlmueller, and Owen Cotton-Barratt.
objective abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain abstain abstain objective objective objective objective objective objective objective objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain method abstain abstain abstain method abstain method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain result abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain method abstain abstain method other abstain other other other other other other other other other other abstain abstain abstain objective abstain result abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain other other