In fact, I don’t see such a difference with other models like OpenNMT-py 7
I’m implementing my model as part of onmt so the train.py is the same.
The thing is that

my model has less parameters than default ONMT
my model uses more history variable (e.g. previous states h_t, c_t etc.)

Maybe the second point makes it more difficult to calculate gradient?

For reference I’m working on the ML part of Paulus et al (2017) 8