Solved by ptrblck in post #7 
              
              
                Thanks for the code!  
I got an error of a device mismatch, since hidden_state and cell_state are both initialized on the CPU even if you push the model to the GPU. 
Could you try to use: 
    def forward(self, X):
        X = self.embedding(X)
        
        trans_X = X.transpose(0, 1) # Make it …