Is there any convenient way to train a transformer from scratch ?
Hello guys!
`Huggingface/transformers` is such a convenient library to use when it comes to all sorts of pertained model. But I am wondering is there a convenient way to train a model from scratch ?
If I want to rebuild the model in `Attention is all you need` , the first thought came into my mind is change modeling_bart.py to adapt to`Attention is all you need` setting like `three way weight tying`,,and do not using `.from_pretrained`, 
Is there any better way to do it ?
I know this is a **pertained_model** library , but isn't is be cool to do something more with it?
I am looking forward to your reply
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

There are a few scripts related to pretraining: [language-modeling examples](https://github.com/huggingface/transformers/tree/master/examples/pytorch/language-modeling)

Thanks!