ru language
which pre-trained model can work for russian language? i want get only vectors
of course i can use bert aka multi-language but they work very bad  in my mind for ru <|||||>I think XLM models is better than mBERT. There are XLM models for 17 and 100 languages including ru.<|||||>cool
can u help me where i can download pre-trained xlm from ru because [here](https://huggingface.co/pytorch-transformers/pretrained_models.html) i can't find models for RU?<|||||>i think [here](https://github.com/facebookresearch/XLM) i can get pre-trained<|||||>@vtrokhymenko Yes, it's here<|||||>While wanting to understand how to convert a BERT Tensorflow model to one that works in pytorch-transformers, I stumbled upon RuBERT from DeepPavlov.

https://github.com/fredriko/bert-tensorflow-pytorch-spacy-conversion<|||||>or this 
```
import tensorflow as tf
  
from bert_dp.modeling import BertConfig, BertModel
from deeppavlov.models.preprocessors.bert_preprocessor import BertPreprocessor


bert_config = BertConfig.from_json_file('./rubert_cased_L-12_H-768_A-12_v1/bert_config.json')

input_ids = tf.placeholder(shape=(None, None), dtype=tf.int32)
input_mask = tf.placeholder(shape=(None, None), dtype=tf.int32)
token_type_ids = tf.placeholder(shape=(None, None), dtype=tf.int32)

bert = BertModel(config=bert_config,
                 is_training=False,
                 input_ids=input_ids,
                 input_mask=input_mask,
                 token_type_ids=token_type_ids,
                 use_one_hot_embeddings=False)

preprocessor = BertPreprocessor(vocab_file='./rubert_cased_L-12_H-768_A-12_v1/vocab.txt',
                                do_lower_case=False,
                                max_seq_length=512)

with tf.Session() as sess:

    # Load model
    tf.train.Saver().restore(sess, './rubert_cased_L-12_H-768_A-12_v1/bert_model.ckpt')

    # Get predictions
    features = preprocessor(["Bert z ulicy Sezamkowej"])[0]

    print(sess.run(bert.sequence_output, feed_dict={input_ids: [features.input_ids],
                                                    input_mask: [features.input_mask],
                                                    token_type_ids: [features.input_type_ids]}))

    features = preprocessor(["Это", "Берт", "с", "Улицы", "Сезам"])[0]

    print(sess.run(bert.sequence_output, feed_dict={input_ids: [features.input_ids],
                                                    input_mask: [features.input_mask],
                                                    token_type_ids: [features.input_type_ids]}))
```