Thank you for the reply. @SnowWalkerJ 
My thoughts were the same, but I’m running into an issue with DataLoader returning non-CUDA tensors, which made me think that that’s not how it was meant to be used. Have you faced this issue ?

  
    
    
    Dataloader returning non-CUDA Tensors 
  
  
    I create a simple dataset that has two CUDA tensors, one of shape (1000, 5) and the other (1000,) but when I load them using a DataLoader only the first one is returned as CUDA tensor. I think this may have something to do with the default collate_fn of the DataLoader. Any fixes ? 
import torch
from torch.utils.data import Dataset, DataLoader
import numpy as np

x = torch.cuda.FloatTensor(np.random.randn(1000, 5))
y = torch.cuda.LongTensor(np.random.randint(0,10, size=(1000, )))

class data(Data…