Hey @jwyao, have you tried setting find_unused_parameters=True in DistributedDataParallel (DDP) constructor?
The error message says that DDP instance didn’t see gradients for parameters 11 12 13 14 15 16 19 20 21 22 23 24.
You can verify this by running one forward + backward and then loop over all parameters and check if there .grad field is available/updated. Sth like:
model = DistributedDataParallel(model, devices=[rank])
loss_fn(model(inputs)).backward()

for p in model.parameters():
  if p.grad is None:
    print("found unused param")

If above doesn’t work, could you please share a min-repro?