When doing multi-gpu training with DistributedDataParallel, the zombie processes will not show up in nvidia-smi. But I found the following command helpful for killing zombie processes,
ps x|grep python|awk '\''{print $1;}'\'' |xargs kill -9