Fine tuning XLMRoberta for Question Answering
# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 

I'm trying to fine tune XLM Roberta for Question Answering in tensorflow version, and the question is do i need to convert the pytorch pretrained model to tensorflow because 'xlm-roberta-case' = "https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-base-pytorch_model.bin", if so how can i do it? i tried to use load_pytorch_model_in_tf2_model() but i had errors!!
 thank you for help in advance.

Hi, we don't currently have an implementation of XLM RoBERTa in tensorflow.<|||||>I guess this one can now be closed since TensorFlow XLM-RoBERTa was released with 2.4.0. Thanks @LysandreJik @jplu . Quick question though: I guess you are not retraining the LM but convert the pytorch weighs. Is there any script in huggingface to do this?<|||||>Right now the easiest way to convert the PyTorch weights to TensorFlow when the two implementations are in huggingface/transformers is the following, for e.g. XLM-R:

```py
from transformers import XLMRobertaModel, TFXLMRobertaModel

pytorch_model = XLMRobertaModel.from_pretrained("xlm-roberta-base")  # Checkpoint on S3
pytorch_model.save_pretrained("pytorch_checkpoint_directory")  # Save it to a directory

tensorflow_model = TFXLMRobertaModel.from_pretrained("pytorch_checkpoint_directory", from_pt=True)  # Load from directory in TF
```

You can then save that TensorFlow model using the `save_pretrained` method, and you can do it the other way around too to conver TensorFlow models to PyTorch models.<|||||>@houdaM97, as @nchocho said XLM-R in TensorFlow was released in v2.4.0 last week. There are no official checkpoints on our s3 however, but there are contributed community checkpoints from @jplu you can use instead:

```py
from transformers import TFXLMRobertaModel

model = TFXLMRobertaModel.from_pretrained("jplu/tf-xlm-roberta-base")
```