Have you checked your weights? Initially Pytorch Tensors have Float type and it will cause problems when you want to use different type of tensors. For example:

For solution, you should change your layer weight type. Also you can set default tensor type if you are using all LongTensor:
torch.set_default_tensor_type('torch.cuda.LongTensor')