By “performance”, I mean the classification accuracy. Somehow DDP+SyncBN achieves worse test accuracy than DP, so there must be some problematic differences in the numeric algorithm. The speed isn’t the issue here. Thanks!