Thanks for your reply. I think you are right. But the problem is that the GPU memory is extremely unbalanced. The first GPU   comsumes a lot of memory while othes only used a little. For example
|    0     22043    C   /usr/bin/python                               11138MiB |
|    1     22043    C   /usr/bin/python                               5724MiB |
|    2     22043    C   /usr/bin/python                               5548MiB |
|    3     22043    C   /usr/bin/python                               5613MiB
Any ideas?