lingvisa:

Does Pytorch automatically make use of GPUs that currently have free memory?


No, the used GPU is specified via its index as given in your code example.



 lingvisa:

Also, the above picture shows that GPU 0 is not used at all, why it also caused out of memory issue?


The output shows that GPU0 is almost completely filled, so I’m unsure what “not used at all” would mean in this context. Since this device has only ~4MB left, I would assume that a new allocation would raise an out of memory error.