• Annotators without machine learning background. The GSM8K dataset (Cobbe et al., 2021)
conveniently provides a training set with reasoning chains written by crowd compute workers,
which enables us to investigate whether chain of thought still works with reasoning chains from an
independent source without a background in machine learning. So we randomly sampled three sets
of eight exemplars with chains of thought from GSM8K. These chain of thought annotations also
outperformed the baseline by a large margin for all four arithmetic datasets (Table 6), indicating
that chain of thought is not dependent on a particular set of annotators.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models