Solved by ptrblck in post #4 
              
              
                Thanks for the code. 
I can reproduce this behavior using two simple DataLoaders: 
X_train = torch.arange(10).float().view(-1, 1)
y_train = torch.arange(10).float().view(-1, 1) + 0.1
train_dataset = dataset_class(X_train, y_train)
train_loader = DataLoader(train_dataset, batch_size=2, shuffle=True)
…