Do you use manually tracked tensors in your network? For example in the .forward() method. If you use a tensor alone, without wrapping it in a torch.nn.Parameter, it won’t be “part” of the network, and won’t be mapped to GPU when you call .cuda().