I converted a colab.research.google.com Jupyter notebook to run on my local machine.  Unsurprisingly it takes much longer.  However, when cProfiling the python code I find that 40% of that time is taken in torch.Tensor.cuda() - which is basically copying your objects from RAM to memory on the GPU.  Any way of speeding this up short of “buy a new faster machine”?  Buy a new faster machine that is, with a different and faster interconnect between RAM and the GPU.  Discrete (I assume) GPU card - GeForce GTX 1660.  New machine - haven’t opened it up and looked around much.
Interestingly, I have a routine that measures TFOPS and the 1660 is faster by some 1/3 than whatever the default GPU is that I’m getting in an unpaid Colab acct.  Wonder if paying the $9.99/mth results in faster GPUs or interconnects?  Although of course that’s not where my current bottleneck is.  The two training steps of the code in question take 1.5 hrs on Colab - and much longer locally.