(1) In fact, I did run without CUDA_VISIBLE_DEVICES.  The machine I’m using only has one GPU and when I run without setting CUDA_VISIBLE_DEVICES, the machine automatically knows to use the only GPU as cuda:0.
(2) I’m not sure I understand this quote

I assume the load method is smart enough to copy the device state_dict back to your CPU model.

Actually I am trying to get the map_location argument to load the model from disk, trained on the GPU, directly into the GPU’s VRAM (as advertised in the documentation).  Without having to go through CPU.  So I thought it should not end up on the CPU.
(3) Thank you for the suggestion.  Since my goal is just to understand and I’m wary of nightly binaries, I will pass on them for now.  But to clarify, you are saying, in the newest version, my code above automatically loads the model to the GPU’s VRAM without first loading into CPU memory then transferring to the GPU?
Thanks.