thanks @ptrblck,
I tried the code, I still get:
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.88       Driver Version: 418.88       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  On   | 00000000:0C:00.0 Off |                  N/A |
| 44%   69C    P2   247W / 250W |   2579MiB / 11176MiB |     98%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 108...  On   | 00000000:0D:00.0 Off |                  N/A |
| 28%   32C    P8     8W / 250W |     10MiB / 11178MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

The second gpu still shows only 10 percent usage…
also, my model is not training very well, when training on multiple gpus using data parallel, how should i change my learning rate?