Solved by Dennis_Wu in post #5 
              
              
                Okay, i just solved the problem by myself, the reason of this is the Attn() function which i wrote outside the model class as another def() function, and the Attn() function will not be moved to the GPU, so I create a new nn.Module class for Attn and i wrote : self.attn = Attn(hidden_size) in the mo…