The problem in my case seems to be that the handling of variable sized tensors (due to variable sequence lengths). I assume then that this is internally implemented with a loop. I will look into scripting but if the loop is in the pytorch code then how would I do it? Another alternative that I just considered is inputting the hidden state for the GRU and passing it to the network which will output the hidden state for use in future steps. That way I suppose I could use use a dummy with length 1 and perform the loop outside. Would something like this work?