Thanks for responding.
I’ve modified (hacked) the following so that it will run locally:

  
      
      colab.research.google.com
  
  
    

Google Colaboratory



  
  
    
    
  
  


If need be, I could post my version somewhere.
Then I run locally with:
ipython -m cProfile FlowerML-2.py
As I was googling on the problem I noticed your id reply to someone somewhere to the effect that asynchronous operations were being added to pytorch (probably long ago) but that the person asking would need to build pytorch for him or herself.
… ah, found it (your earlier conversation):

  
    
    
    Does torch.Tensor.to supports async GPU transfer? [SOLVED: yes] 
  
  
    It all in the title: 
torch.Tensor.cuda has an async parameter that allows asynchronous transfer from pinned memory to GPU. Does torch.Tensor.to automatically uses async when possible? 
I wasn’t able to find the relevant piece of source code.
  


June 2018.
I’m a little confused - immediately after the call to .cuda() the same objects are operated upon, presumably on the GPU and happening under-the-hood within pytorch, and so I was assuming this would need to happen synchronously.
Not all of which (the above) necessarily addresses how you’ve asked your question.  I’m not at all new to programming but relatively new to ML and pytorch.