This is not a Pytorch problem. It’s a problem with Python multiprocessing, sometimes, python workers wil not die if the python script fails. So all you need to do , is find these zombie python processes and kill them.
You can do this like so:
ps -elf | grep python
kill -9  [pid]

with pid being the process id of each of the python processes that the first command outputs. You can also use killall python. But I recommend the first option.