Plan to support UniLM ?
# 🌟New model addition

## Model description

**UniLM** : Pre-trained transformer for sequence to sequence generation.

Paper : https://arxiv.org/pdf/1905.03197.pdf

## Open Source status

* [x] the model implementation is available: **[official Pytorch](https://github.com/microsoft/unilm)**
* [x] the model weights are available: For now only english : [UniLMv1-large-cased](https://github.com/microsoft/unilm#pre-trained-models)

## Additional context

The official implementation is based on a modified version of this repository (version 0.4.0).  
That would be nice to have a unified API :)

*Note : They didn't release the code for pretraining yet.*
This is on our mid-term roadmap.
We have a project adding Seq2seq models and UniLM will be part of this project.