Just to keep you updated about this: I used the nightly version with zero_infinity flag set to true but I am still getting nan from a certain time step on.
What seems strange to me is that if I use a constant lengths_input (torch.full) this never happens.
Of course in this way the length of the input stays the same or it is increased so I reduce the number of invalid samples, but by zeroing the inf loss I should get the same result.
I will continue to try to fix it and eventually let you know.