In this case it would help if nn.Linear had a requires_grad option in its interface
It could be passed to Parameter() call while making weights 2 and biases.
@albanD @SimonW. Does this seem useful ? Can we implement it? Will it cause any problems to the existing code-base?
@albanD Your solution with in-place copy worked correctly after setting requires_grad=False for Linear layers of C