Thanks for the quick reply! I actually added that ‘device=‘cuda’’ part in the forward during troubleshooting and forgot to remove it, but the same problem persists when I remove that. Is it mandatory that whenever I make a new tensor in the forward that I set the device, or is DataParallel doing that for me?