Solved by ptrblck in post #7 
              
              
                The loss.backward operation isn’t sped up or slowed down by the usage of another Dataset, but depends on the model architecture as well as input shapes. 
If you’ve increased the input shapes in the new workflow, a slow down might be expected. Otherwise, I would guess you are timing your code wrong. 
…