You can write a dataset class where in the __init__ function, you red the entire dataset and apply all the transformations you need, and convert them to tensor format. Then, send this tensor to GPU (assuming there is enough memory). Then, in the __getitem__ function you can simply use the index to retrieve the elements of that tensor which  is already on GPU.