Some weights of AlbertModel were not initialized ['albert.embeddings.position_ids'] 
Hello!

There seems to be a problem with the current code to load a pre-trained Albert model. This warning appears in any configuration of the Albert model:
`Some weights of AlbertModel were not initialized from the model checkpoint at albert-base-v2 and are newly initialized: ['albert.embeddings.position_ids']`
`You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.`

I found this happens only when I install it from the source. Models load correctly (without warning) when installing the library with pip.
Hello! Which checkpoint are you trying to load? Is it one of your checkpoints or is it one of the checkpoints hosted on the modelhub?<|||||>This is probably because of this PR: https://github.com/huggingface/transformers/pull/5773 , but should not pose a real problem. I guess we just have to add the position ids as "allowed" non-initialized weights.<|||||>@LysandreJik I used model hub checkpoints. I runned the following lines with pytorch:

`from transformers import AlbertForPreTraining`
`model = AlbertForPreTraining.from_pretrained('albert-base-v2')`
<|||||>Hey @vgaraujov, I can reproduce - this PR: #6700 should fix it. Thanks for reporting it :-) <|||||>@patrickvonplaten Thank you for your help! 💯 <|||||>Position_ids seems unnecessary to be saved? Why not use register_buffer with persistent=False<|||||>> Position_ids seems unnecessary to be saved? Why not use register_buffer with persistent=False

It's a fantastic suggestion, @ruotianluo!

But, alas, It can't be used at the moment, since:
1. this functionality  [was added just a few months ago](https://github.com/pytorch/pytorch/pull/37191) (can't require recent `torch`)
2. [it doesn't yet work with torchscript](https://github.com/pytorch/pytorch/issues/45012)

See the following solution: https://github.com/huggingface/transformers/pull/7224