FSMT Training scripts
Hi, thank you very much for your implementation and it helps a lot. I want to train the MT models from scratch. Could you provide some instructions about the training details for FSMT? Thanks a lot. @stas00 
Thank you for the kind words, @wangyong1122.

The initial task was to port the weights so that it could be used for Translation as is, which is mostly done. I'm still working out some kinks under CUDA - I forgot to test with USE_CUDA=1 :( But I'm almost there.

I'd say, how about you start the training - like you'd any other `transformers` model that you already have the instructions for and I'm 100% sure you will encounter problems since that part wasn't worked on/tested at all. So when you run into problems please post the code so that I could reproduce the issues and slowly slowly we will sort it out. How does that sound?

It's my first model ported to `transformers` so things take much longer as I'm learning the nuances of this platform.<|||||>Thanks very much for your kind reply, @stas00.
It sounds great! If I run into problems, I will post the code.<|||||>To explain why I expect problems: `transformers` has been written with a single merged vocab in mind, whereas fairseq's transformer has 2 distinct vocabs of different size (for en-ru/ru-en, but merged for en-de/de-en). So I had to do all kinds of hacks to make things work - for example to overcome `self.vocab_size` usages in core functions, as here we have `self.src_vocab_size` and `self.tgt_vocab_size`. 

So for the initial task of making translation work - almost everything should be fine, as most of it is well tested. 

But for training from scratch and finetuning I expect problems - so we will solve them gradually as you uncover them. 

The best help you can provide is to code those problems in new tests. If you can't, that's ok too, this is just if you'd like an extra learning challenge. But in any case, please, make small simple reproducible examples - train on 3 lines of text that you can hardcode here, and not 3GB that need to be set up :)<|||||>Thanks very much for your details. @stas00.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
