albanD:

nn.Parameter (that is basically a Tensor that always require grad)


But it still does allow setting require grad to False. (link 2)
Do you know why this is provided then?



 albanD:

If you go through the process of setting it’s weights by hand, you can also set the requires_grad to False in that process without any extra complexity.


This seems right, it’s negligible overhead, but still setting it during intialisation wouldn’t hurt
Wouldn’t this be useful for hypernetworks, networks that generate weights for other networks?