Hello,
I am facing an issue when back-propagating the loss due to large batch size due to the accumulation of the 8 GPU DataParallel. Is there is a way to divide the loss calculation into 8, then normalize the loss then use the loss.backward() but this time it should be smaller in size or how i can work this around ?