I’m not familiar with trax and would recommend to check smaller code blocks for the right implementation including the gradients.
I.e. isolate layers and compare the output as well as the gradients for both frameworks using a constant input and the same parameters.
Since you are dealing with LSTM modules, I would start with them and make sure the dimensions are the same in both implementations.