My training sometimes stucks oddly. It stops printing.
image747×59 2.14 KB
And the GPU utils is 0%.
image877×84 3.43 KB
When I press Crtl + C to terminate it, it prints log like this.
image1267×513 25.9 KB
It seems that the error occurs when loading data by multiprocessing.
Anyone knows why?
The verison of some modules in my code:
python version：3.7.9
torch version：1.4.0
cuda version：10.0
I use the docker interactively, and use screen to run in background.