PyTorch version from 1.9 to PyTorch 1.4 and the issue gets resolved.

In that case I would bet on the optimizer doing the inplace update on a weight and you’re trying to backward again after. But the weight was modified inplace by the optimizer hence the error.
Versions before 1.8 (IIRC) had buggy optimizers for which the inplace was not registered properly and it was silently wrong.
You should make sure that you never do forward then optimizer step then backward. You need to redo the forward after an optimizer step before doing a backward.