thanks for your reply. To my understanding, I could not find any typical problematic in-place operation in the network structure (  it’s working in non-distributed mode) I don’t know if there exist new issues with in-place operation in DDP mode.  for the optimizer, I’m using the classic way mentioned in tutorials.