Hi yall, Ive been doing hyper-parameter training and in my case found that the cost of evaluating various learning statistics and breaking epochs early was fine. But when my hyperparameterizer was doing that I found errors leading to multiprocessing.py - I presume that my function started a new process before the old one was finished or something of the sort.
I looked at dataloader.py and did not find a method that joins the queues to prevent this error. So I just hacked a time.sleep(arbitrary_n_seconds) and then call trainloader.iter()._shutdown_workers()
So, question, is there such a method built into DataLoader or shouldnt there be one? It seems like simple thread management.
pytorch 0.4.0 from conda
python 3.6.5
ubuntu 16.04
no difference if device is cpu or gpu