in the OpenAI API). We sample from the models via greedy decoding (though follow-up work shows
chain-of-thought prompting can be improved by taking the majority ﬁnal answer over many sampled
generations (Wang et al., 2022a)). For LaMDA, we report averaged results over ﬁve random seeds,
where each seed had a different randomly shufﬂed order of exemplars. As LaMDA experiments
did not show large variance among different seeds, to save compute we report results for a single
exemplar order for all other models.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models