Hello! Have you solve the problem? My code also stucks with DistributedDataParallel during backward. And it always stucks at the same iteration. I don’t know how to diving into backward debugging. Hope you can provide the solution. Thanks!