Getting embedding from XLM in differnet languages
## ❓ Questions & Help

Hi,

I'm trying to get a cross-lingual embedding from the XLM model, but can't figure out how.
In the project original github, you need to give the tokenizer the language of each of the tokens, but it doesn't seem the case here.
Will appreciated any help on the matter :)
Hi, are you referring to the sentence embeddings that are generated in the [XLM notebook](https://github.com/facebookresearch/XLM/blob/master/generate-embeddings.ipynb)?<|||||>Yes. Although I'm using it to get the word embeddings (see [here](https://github.com/facebookresearch/XLM/issues/17)).
Maybe I'm missing something, but as far as I understand, the model uses a language embedding  that is added to the token embedding, so it seem it will need that information. Am I missing something?<|||||>Well, neither the official XLM notebook @LysandreJik linked to nor the XLM repo issue @OfirArviv linked to are mentioning the need to give language ids so I'm not sure exactly why they would be needed.

Maybe this is a question for the original authors of XLM?<|||||>Hi @thomwolf!

I believe they already answered this question in [this](https://github.com/facebookresearch/XLM/issues/103#issuecomment-501682382) [issue](https://github.com/facebookresearch/XLM/issues/103#issuecomment-501682649): 

So it will be useful if we can provide models with lang ids, preferably during training as well.

  
<|||||>Ok I see.

So you need to input a `torch.LongTensor` with the `language id` for each token in your input sequence in the model (see inputs [here](https://huggingface.co/pytorch-transformers/model_doc/xlm.html#pytorch_transformers.XLMModel)).

Right now the conversion mapping from language to ids (and vice-versa) can be found in the configuration of the model (see for ex [here](https://s3.amazonaws.com/models.huggingface.co/bert/xlm-mlm-xnli15-1024-config.json)).

Here is an example:
```python
from pytorch_transformers import XLMModel
model = XLMModel.from_pretrained('xlm-mlm-xnli15-1024')
lang2id_dict = model.config.lang2id
id2lang_dict = model.config.id2lang
```

If you only want the conversion dictionary and not the model, just load only the configuration:
```python
from pytorch_transformers import XLMConfig
config = XLMConfig.from_pretrained('xlm-mlm-xnli15-1024')
lang2id_dict =config.lang2id
id2lang_dict =config.id2lang
```

I'll add more details on that in the docstring.