No language embedding weights in pre-trained xlm models. 
I'm trying to train a one-shot classification model using the given XLM pre-trained weights.  
However, I noticed that for both `xlm-mlm-17-1280` and `xlm-mlm-100-1280`, That I kept receiving the warning `weights of XLMForSequenceClassification not initialized from pre-trained model: ['lang_embeddings.weight']`. I then looked into the state_dict of those two checkpoints and saw that indeed there were no weights that matched that key. Should that weight exist in the state_dict somewhere?
Indeed, I've checked and the 100 and 17 language models don't use language indices.
Just supply `langs=None`.
You can see that in the official notebook from Facebook: https://github.com/facebookresearch/XLM/blob/master/generate-embeddings.ipynb<|||||>I see. I missed this issue here which explains it pretty well. https://github.com/huggingface/pytorch-transformers/issues/1034
Thanks for the help!