Unfortunately, the source code depends on different modules and large data, it’s not useful for debugging purpose. In addition, the code perfectly run on other server when I connect to it via ssh and directly run my python.
The new server is based on Kubernetes and OpenShift and my code is deployed via a docker container. I think that it causes the misidentification of GPUs by DataParallel.
Did you have any related experiment?