Solved by Nasi_Kaashki in post #3 
              
              
                Thank you. Good suggestion. Having debugged the code and watched the tensors, I have noticed that rand_grid is on the different GPU which affects y as well. By fixing that, the problem got solved. 
I replaced self.vertex = torch.from_numpy(point_set).cuda().float() with self.vertex = torch.from_nump…