In terms of the discrepancy in loss across DDP and DP, does this happen from the first iteration itself or it slowly creeps in where after a large number of iterations you see some deviations? Also, what is the percentage difference between the loss you see for DDP and DP at the end of training?