yoad.tewel:

From my experience, usually the first epoch of training (for any dataset) is slower, but after that python/OS uses some caching mechanism, and reading images from the same path as before is much faster, thus training is much faster.


That’s an interesting observation I haven’t seen so far. Do you have any references to it?
For a general overview of data loading bottlenecks and potential workarounds I would refer to this post 97.