You don’t need to wrap the subsets again in TensorDatasets.
In fact, you would create your Dataset first (e.g. as a TensorDataset or any other) and wrap it into a Subset to get the samples for the corresponding indices. This subset can then be wrapped by a DataLoader.
Note that TensorDataset is used for tensors already loaded into memory.
If you are dealing with a large dataset, this might not be possible, so you could use another dataset, e.g. torchvision.datasets.ImageFolder or a custom Dataset implementation for lazy loading.



 blackbirdbarber:

I am not sure if I can use something other than TensorDataset , but the idea was to
to move tensors on CUDA.


The usual way would be to load the samples into the RAM and push each batch onto the GPU in the training loop.
If your model is small and you are not working with a lot of data samples, you could of course push the whole dataset onto the GPU beforehand, but note that you might be wasting precious GPU memory.