My GPU usage:
# nvidia-smi
Fri May 21 13:31:47 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.87.01    Driver Version: 418.87.01    CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  On   | 00000000:00:10.0 Off |                    0 |
| N/A   33C    P0    56W / 300W |  16126MiB / 16130MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla V100-SXM2...  On   | 00000000:00:11.0 Off |                    0 |
| N/A   33C    P0    57W / 300W |   1517MiB / 16130MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  Tesla V100-SXM2...  On   | 00000000:00:12.0 Off |                    0 |
| N/A   39C    P0    56W / 300W |   1519MiB / 16130MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  Tesla V100-SXM2...  On   | 00000000:00:13.0 Off |                    0 |
| N/A   55C    P0   278W / 300W |  15965MiB / 16130MiB |     97%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

x_data = torch.tensor([[1, 2],[3, 4]])
x_data.to(“cuda:0”) and x_data.to(“cuda:3”) caused:
RuntimeError: CUDA error: out of memory
‘cuda:1’ and ‘cuda:2’ are fine without memory error.
Does Pytorch automatically make use of GPUs that currently have free memory? Also, the above picture shows that GPU 0 is not used at all, why it also caused out of memory issue?