Solved by Nikronic in post #4 
              
              
                Hi, 
You should not load whole datasets into your RAM if you do not have enough RAM to do that. You should use lazy loading provided in DataLoaders in PyTorch. As you mentioned, You have to prepare next data when your GPU is processing. 
By the way, This is best and most efficient solution to task s…