Documentation for tiny-gpt2 in transformers/examples/pytorch
# 🚀 Documentation request

The **tiny-gpt2** transformer model is great for fast prototyping, but it seems sparsely documented on the Huggingface hub: https://huggingface.co/sshleifer/tiny-gpt2

## Motivation

It would be helpful if users knew basic info about how tiny-gpt2 was trained. Is it the same corpus as the standard gpt2? Was it distilled from a larger model or trained from scratch? Etc.

## Your contribution

As I did not train tiny-gpt2, I don't know any info about it.
It is randomly initialized and trained for 2 steps. So basically it can only be used for prototyping.