But as long as there is some nn.Parameter the result will require grad so not sure why is that necessary?
By accomulating gradients I mean several
forward and backward passes without calling zero_grad between them.
The gradients are accomulated in .grad field.
It’s used for example to simulate bigger batches (together with loss scaling).