Usually you would like to avoid long startup times and thus push the data loading and processing to the __getitem__ method. If you are using a DataLoader, multiple workers can load the data in the background while your GPU is busy training, which might hide the loading time.
Currently you are only timing one epoch, which also doesn’t consider the initialization of your Dataset.
However, if you just would like to slice the pd.DataFrame without any fancy indexing, it might indeed be faster to transform it to a tensor beforehand.