The training step parts of “run backward” and “optimizer step” are independent in PyTorch, so the first computes gradients for everything that has requires_grad=True and is ignorant of whether you use them in the optimizer step. So you would keep allocating gradients for the unused parameters but never use (nor zero) them.
Best regards
Thomas