Mbart continue training with same training task on a specific language
Hello.

I am not sure if this is possible using the transformers library, but it isn't it would be nice to have.

Mbart was initially trained using span corruption and other training tasks on a corpus containing many languages. Since I am going to use it for specifically the Arabic language, I wish to solely finetune it on Arabic text with the same training tasks it had, only then at a third step to fine tune for specific text generation task.

I believe this is possible using fairseq, but having it here in the transformers library would better. Is that possible / useful per your opinion ?
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!<|||||>@LysandreJik  Sorry, I will close the issue now. 
However this might qualify as a useful feature request, don't you think?