Thanks for the response. I have a couple questions around this:

Yes, the code I am using needs to spawn more processes than the number of cpu cores it gets allotted. I thought this is okay (and I may not be speaking with a lot of knowledge about this) if the kernel just shares the load among these processes like any others. Is this something inadvisable, i.e., should num_workers over all loaders be less than the number of cpu cores?
Is set_num_threads the best way to restrict the program to a certain number of cpu cores on a server?
I think file reads do not need any locks, and multiple processes can read from a file. Any reason you can think of why ImageFolder might behave differently from FakeData? (I am currently trying out FakeData on my end just to record the result)

Just another update : I tried calling del() on the iterators explicitly before reinitializing them (to try to make sure the workers exit properly), and I still get the program stuck at some point.