Hi, I’m using hooks for validation, and with data parallel only the packet of gpu=0 returns, I saw that there is not advance with that issue.
So I thought on a hack, I will train with data-parallel but when doing validation I’ll remove the data parallel, and send the model to 1 gpu only.
No matter what I tried to do it did not succeed, “to(‘cuda:0’)”, sending it to cpu, no matter what it returns half packet.
So now to my question, how can I take a model with data-parallel wrapper, and remove the data-parallel wrapper of him?!!?