Hi!
I am using a nn.parallel.DistributedDataParallel model for both training and inference on multiple gpu.
To achieve that I use
mp.spawn(evaluate, nprocs=n_gpu, args=(args, eval_dataset))
To evaluate I actually need to first run the dev dataset examples through a model and then to aggregate the results. Therefore I need to be able to return my predictions to the main process (possibly in a dict, but some other data structure should work as well). I’ve tried providing an extra dict argument mp.spawn(evaluate, nprocs=n_gpu, args=(args, eval_dataset, out_dict)) and modifying it in the function but apparently spawn copies it, so the dict in the main process is not modified.
I guess, I could write the results to the file and then read in the main process but it doesn’t seem like the most elegant solution. Is there a better way to return values from spawned functions?
Thanks!