Inconsistency between GPTNeo and GPT2 config classes
The config classes for GPTNeo and GPT2 have a bunch of differences that are seemingly unnecessary. This makes it harder for downstream users to write code that depends on accessing these attributes. See below:

![image](https://user-images.githubusercontent.com/54557097/122113739-bbf92300-cddf-11eb-9ac6-a0ea1a30055b.png)
It seems that max_position_embeddings, hidden_size, num_layers, num_heads, intermediate_size, resid_dropout, embed_dropout, and attention_dropout should be renamed for sonsistency with the GPT2 config class.

### Who can help
@LysandreJik @patil-suraj 
Seconding this.

Last month I swapped out GPT-2 for GPT-Neo in a [project](https://github.com/nostalgebraist/nostalgebraist-autoresponder/), and these differences made it more difficult to adapt my existing code.<|||||>Hi @leogao2 and @nostalgebraist, thanks for opening an issue! You're correct that the way this is currently implemented it prevents a few use-cases. Namely this is authorized:

```py
from transformers import GPT2Config

config = GPT2Config()
config.hidden_size
```

But these are not:

```py
from transformers import GPT2Config

config = GPT2Config()
config.hidden_size = 4
# Fails

config = GPT2Config(hidden_size=4)
# Fails
```

Unfortunately we can't just rename arguments - as this would break both checkpoints on the hub as well as local checkpoints. We're thinking of a way to enable this with a convention set across configurations for the attributes you mention - this convention would allow getting and setting attributes that are defined in this convention, such as the ones you mention.

Let us explore a bit and we'll come back to you. cc @patil-suraj @patrickvonplaten @sgugger <|||||>> This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
> 
> Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

This still needs to be addressed.<|||||>Is there any progress on this?<|||||>Hey @leogao2! Yes, a proposal is available here: https://github.com/nreimers/transformers/commit/2198ee719c6101ef47de00cd6d53da3a8f938fb4 but there are still a few rough edges to polish. We'll try to have it merged in the next few weeks, will let you know.<|||||>This was fixed in #13026 which will be in the next release alongside GPT-J. Thank you for opening an issue!