GPT2 medium config n_ctx is wrong I guess?
Hi Guys,
For gpt2-medium ```n_ctx: 4096``` right ?
But in config it is showing ```n_ctx: 1024``` . 
No, 1024 is correct. It refers to the sequence length of the model. <|||||>Oh sorry.
Then whats the config parameter of ```intermediate projection after attention```, which is ```4096``` in gpt2-medium.<|||||>Looking at the [config attributes of GPT-2](https://huggingface.co/transformers/model_doc/gpt2.html#transformers.GPT2Config), there's an attribute called `n_inner` which is defined as "Dimensionality of the inner feed-forward layers. None will set it to 4 times `n_embd`". 

Apparently, the `n_embd` attribute of the medium-sized GPT-2 model is 1024. So this times 4 equals 4096. <|||||>Oh my bad. Thanks. There is too much of inconsistencies between different model configs. 