Toy size models versions for faster experimentation and testing
# 🚀 Feature request

To have a small(toy) in terms of number of parameters versions of models e.g.:
- `gpt2-toy`
- `bert-toy-uncased`
etc.
Alternative:  to have separate model type `toy-transformer`.

## Motivation

Working on the project and using transformers package our team noticed that it's difficult for some team members who had machines without GPU to experiment with code working with GPT2 model and to run tests for it. Also, it required significant resources on CI environment to run tests.

We've ended up by implementing a custom configuration for GPT2 model with a resulting model size ~10M parameters for local experimentation and testing.

I think it might be beneficial for the community to have tiny model versions in the package that weakens hardware requirements to start work and to run tests.
If it's unreasonable to implement tiny version for every model - having separate model type which implements same interfaces and have corresponding heads will also help to reach the same objective.

## Your contribution

If either of the options sounds reasonable I can participate in the implementation.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
