A question about BERT position embedding.
## ❓ Questions & Help

I noticed that in Transformers, it turn position_ids into position_embedding through `nn.Embedding(config.max_position_embeddings, config.hidden_size)`

Here `config.max_position_embeddings` is 512 ,and `config.hidden_size` is 768.

So, when I input a sentence shorter than 512 ,such as "Today is a nice day", will this sentence's position embedding still be 512?  

Or just as long as position_ids? Here is 7 with [CLS] and [SEP].

The sentence "Today is a nice day" will already be padded to 512 because of the tokenization process, so the input will remain 512 in the embedding layer<|||||>> The sentence "Today is a nice day" will already be padded to 512 because of the tokenization process, so the input will remain 512 in the embedding layer

Thanks a lot! 