Support for running Gpt-Neo 2.7B with 6 GB vram for inference
# What does this PR do?
It adds functionality to allow gpt-Neo 2.7B to run in 6gb vram.

If it detects that some modules are on gpu and there is not enough vram, a dict called extrastorage is created which holds the data for model.transformer.h
These weights are loaded from ram to vram one at a time, reducing vram usage.

Expected speed is around 1 token/2s. (slower on the first run)

## Usage
1. Have between 5 and 9.5 Gb Vram

2. run -
```
model.eval().half().to("cpu")
model.transformer.wte.to("cuda")
model.transformer.wpe.to("cuda")
model.transformer.ln_f.to("cuda")
model.lm_head.to("cuda")
torch.cuda.empty_cache()
```
3. Use
`model.generate()
`or 
`model(**inputs)
`

## Motivation
Will become faster as ram->vram (pcie) bandwidth increases
Running larger models on consumer hardware is important

## Incomplete
I need some help with the documentation, also I'm not sure if `import copy` should be inside an if statement or not (line 769).

## Before submitting
 
* [x]  Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
       Pull Request section?
* [ ]  Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
       to it if that's the case.
* [ ]  Did you make sure to update the documentation with your changes? Here are the
       [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
       [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
* [ ]  Did you write any new necessary tests?
 
 
 ## Who can review?
 
 Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
 members/contributors who may be interested in your PR.


Models:
gpt-neo: @patil-suraj
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.