The paper conducts just fine-tuning the pre-trained model, so it seems sufficient. Also, the paper mentions that they implements code with pytorch huggingface implementation although the author doesn’t publish their code yet. I think the project is simple enough . Also, I expect the jit in jax could additionally improve the generation performance compared to PyTorch’s implementation.