Try to (safely) use multiple threads for loading. The torch DataLoader already has multi-threading functionalities and might prove useful. If you train on a GPU, the loading and pre-processing of data in main memory by the CPU adds almost no overhead since it is hidden by the GPU training (i.e. as an iteration happens on the GPU, the CPU loads the next batch).
When it comes to large datasets that won’t fit in your RAM, a memory mapped database can help. Perhaps try to store your (processed) data in an LMDB (Lightning Memory-mapped Database) instance.
EDIT: If you save your data in a database, using DataLoader will be much easier. Large text files are hard to work with since reading is usually done line by line for efficiency and random line reads are slow.