That explains the long loading time.
Do you have a (small) SSD where you could put the data on? If not, you can’t really speed up the loading process, since your magnetic drive is the bottleneck here.
Loading all the data into RAM would speed up the process after an initial loading time, but as you said the dataset is too large to fit into memory so you are stuck with lazy loading.