This is weird, even if pre-training doesn’t help, it should perform at least similar or better than Marian  given the large architecture. Could multilinguality be the reason for this perf drop ?


dynamic batching will be a great addition if it gives speed-up. Can’t fine-tune large models reliably on colab. I want to get bart/MBart working on TPU with Seq2Seq trainer, very hard to get interesting results with large models without huge compute.


I knew it