Do Hugging Face GPT-2 Transformer Models Automatically Does the Absolute Position Embedding for Users?
Hello,

According to Hugging Face ```GPT2DoubleHeadsModel``` documentation (https://huggingface.co/transformers/model_doc/gpt2.html#gpt2doubleheadsmodel)

```
"Indices of input sequence tokens in the vocabulary. 
GPT-2 is a model with absolute position embeddings"
```

So does this mean that, when we implement any Hugging Face GPT-2 Models (```GPT2DoubleHeadsModel```,```GPT2LMHeadsModel```, etc.) via the ```model( )``` statement, the 'absolute position embedding' is _automatically_ done for the user, so that the user actually does not need to specify anything in the ```model( )``` statement to ensure the absolute position embedding?

If the answer is 'yes', then why do we have an option of specifying ```position_ids``` in the ```model( )``` statement?

Thank you,


Indeed, as you can see from the source code [here](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_gpt2.py#L414-L417), when no position ids are passed, they are created as absolute position embeddings.

You could have trained a model with a GPT-2 architecture that was using an other type of position embeddings, in which case passing your specific embeddings would be necessary. I'm sure several other use-cases would make sure of specific position embeddings.<|||||>Ooohh, ok,

so to clarify, absolute position embedding _**is automatically done**_ by the ```model( )``` statement, but if we want to use our custom position embedding (i.e. other than the absolute position embedding), we can use the ```position_ids``` option inside the ```model( )``` statement......is what I said above correct?

Thank you,<|||||>Yes, that is correct!<|||||>Thank you :) !