I wouldn’t say “intentional” behavior, but it is expected behavior.
This doesn’t really have to do with PyTorch, whenever you load that data, it’s all loaded into RAM. What you need to do is implement lazy loading for your data.
An easy way of doing this would be to only load one file a time, train on that data, load the next file, and repeat. However, this may not be ideal depending on your task. Your model may become bias to this order (e.g. training a model on dog pictures only and then cat pictures).
The better solution may be to save all your data into a format that allows lazy loading. hdf5 may be a good contender.