ptrblck:

This subset can then be wrapped by a DataLoader .


You suggest I put a Subset to Dataloader but I just used Dataloader to provide data for the Subset.
I have few problems, with that, I never saw lazy loading, and I am not sure what is this exactly. Searching on PyTorch website gave me 0 results.
dl = DataLoader(
  torchvision.datasets.MNIST('/data/mnist', train=True, download=True,
                             transform=torchvision.transforms.Compose([
                               torchvision.transforms.ToTensor(),                               
                               torchvision.transforms.Normalize(
                                (0.5,), (0.5,))
                             ])), shuffle=False)

train_ds  = torch.utils.data.Subset(dl.dataset, range(0, 50000-1))
valid_ds = torch.utils.data.Subset(dl.dataset, range(50000, 60000-1))


tensor = dl.dataset.data
tensor = tensor.to(dtype=torch.float32)
tr = tensor.reshape(tensor.size(0), -1) 
tr = tr/128 # tr = tr/255
targets = dl.dataset.targets
targets = targets.to(dtype=torch.long)

It would be cool if I could use DataLoader (first line of code) to also scale my tensors by 128 or 256 (divide by 128 or 256) for faster learning and set the float dytpe so once I have Subsets I am perfect to use DataLoader again, but this time I guess two DataLoader, one for training and other for validation.