@abhigenie92 As @Erricia mentioned, the GPUs are renumbered since CUDA_VISIBLE_DEVICES is [1, 2, 3, 4, 5, 6] . So basically, 0 is mapped to the physical GPU 1 and 1 to GPU2 and so on. I verified this locally as well, where I had 0 memory usage on all GPUs and then using [0, 1, 2, 3, 4, 5] as the device ids with CUDA_VISIBLE_DEVICES=[1, 2, 3, 4, 5, 6], I see memory usage on GPUs 1-6:
Wed Jul 29 13:15:28 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.126.02   Driver Version: 418.126.02   CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla M40           On   | 00000000:07:00.0 Off |                  Off |
| N/A   44C    P8    17W / 250W |      0MiB / 12215MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla M40           On   | 00000000:08:00.0 Off |                  Off |
| N/A   43C    P0    67W / 250W |    447MiB / 12215MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  Tesla M40           On   | 00000000:09:00.0 Off |                  Off |
| N/A   42C    P0    65W / 250W |    447MiB / 12215MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  Tesla M40           On   | 00000000:0A:00.0 Off |                  Off |
| N/A   42C    P0    67W / 250W |    447MiB / 12215MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   4  Tesla M40           On   | 00000000:0B:00.0 Off |                  Off |
| N/A   42C    P0    68W / 250W |    447MiB / 12215MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   5  Tesla M40           On   | 00000000:0C:00.0 Off |                  Off |
| N/A   44C    P0    66W / 250W |    447MiB / 12215MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   6  Tesla M40           On   | 00000000:0D:00.0 Off |                  Off |
| N/A   41C    P0    66W / 250W |    447MiB / 12215MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   7  Tesla M40           On   | 00000000:0E:00.0 Off |                  Off |
| N/A   36C    P8    18W / 250W |      0MiB / 12215MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+