Transofrms are only invoked when train_loader iterator is invoked in the training loop. This is quite expensive. I dunno if this is intended behavriour?

When do you want to invoke it otherwise? Before iterating? you can do that, but then you have to store a seperate copy of your image dataset (considering that you don’t want to modify your original images because you might want to try other transformations in other projects)
Also, the data transformation is so fast that it really doesn’t cause any bottleneck on typical dataset if you have num_workers>=4 (I benchmarked this in the past)

The train_dataset.train_data is scaled between [-1, +1] shouldn’t that be between [0, 1]. Since the scaling happens automatically via ToTensor() invocation how can we tell ToTensor() to scale between [0, 1]?

If you want [0, 1] range, simply drop the
trans.Normalize(mean=(0.5, 0.5, 0.5), std=(0.5, 0.5, 0.5)),

Maybe It would be nice for the data loader to have flag indicating to load all data in memory if possible?

I think this would kill the purpose of the data “loader” In that case, simply load the dataset into memory and shuffle before each epoch

Is there a more efficient and better way to do shuffling than my naive way? Essentially is there anything equivalent to  numpy.random.shuffle()  in pytorch?

randperm might suit your purpose:
 all_your_images = torch.randn(500, 3, 28, 28)
 shuffleimages =  all_your_images[torch.randperm(500)]