1M
8 layers

2.5M
8 layers

8.3M
8 layers

28M
8 layers

21M
1 layer

33M
2 layers

GPT2-
XL

to me”.
Katie said ”No, I don’t want to get hurt.”
Katie said ”No, I want to be your friend.”
Katie said ”No, I want to be your friend.”...

to me”.
Katie was very scared. She didn’t know what to do. Then, she had an idea. She said to the man, ”Please
don’t take my cat away”.
The man said, ”No, I don’t want you to take my cat away. I don’t want you to get hurt”.
Katie was very sad. She didn’t understand why the man was so mean. She said, ”I’m sorry, I won’t take
your cat away”.
The man smiled and said, ”It’s okay, but you have to be more careful next time”.
Katie nodded and said, ”Okay, I’ll be more careful next time”.
The end.
TinyStories-HowSmallCanLanguageModelsBeandStillSpeak CoherentEnglish?