Hey @nima_rafiee, which PyTorch release are you using? Can you try v1.7+ if you are not using it? This bug is likely fixed by: [v1.7] Quick fix for view/inplace issue with DDP by albanD · Pull Request #46407 · pytorch/pytorch · GitHub 39