Hello.
I’ve read the code data_parallel.py and the implementation of optimizer.step, but I can’t find the code to accumulate the gradient from multi gpu to single one…
Can any give a hint about the implementation ?
Thanks !