if your model has unused parameters, you could set find_unused_parameters=True; if not all output tensors will be used to calculate for loss, DDP <= PT 1.9 can not support the case yet. But we’ve added a fix to support this case and will be released in PT 1.10, you can try this feature in PT nightly build for now.