Next, we assess the capabilities of the different models on three additional types of prompts:

• Factual prompts, which test the models’ knowledge of common sense facts.
• Reasoning prompts, which test basic reasoning abilities, such as cause and effect and elimination.
• Consistency (context-tracking) prompts test the models’ ability to maintain coherence and continuity

with the given context, such as the names and actions of the characters, the setting and the plot.

We report the generated continuations for each model and prompt in three tables (Figure 9, Figure 10 and Figure
11), and color-code them according to their success (green), failure (red), or partial success (yellow).
TinyStories-HowSmallCanLanguageModelsBeandStillSpeak CoherentEnglish?