Sure, I can get back on the relative speeds tomorrow for both training and inference. The model is not for English on any public dataset, though. Vocabulary size is 800k.
Speed is also the relevant factor for us, but integrating it from the example above in an existing Pytorch model took less than an hour, so dev costs are not that high.