Solved by vdw in post #6 
              
              
                Hm, can’t see anything obvious what might cause the performance issues. Just some generic comments: 


Since you initialize h0 and c0 you don’t need to detach them as well. Maybe that’s even bad for the training itself, but here I’m not sure. 


I cannot see the point of the lines below. Again, I do…