Solved by rwightman in post #5 
              
              
                HDF5 usually does a decent job of caching and handling IO. But if each item you’re fetching and feeding to your NN is quite small, and the net itself is also on the small side, you can still wind up in a scenario where most of the work is IO and Python bookkeeping/data munging and your GPU doesn’t h…