DP will all gather NN result to main device and then all operations are doing in it, that may contain calculate loss get grad and update model param.
DDP keep result on each device until get grad, and do you say average then update model on each device.