@ptrblck Thanks for the explanation and the extended example.
Okay, I get how the detach solution was meant here. Of course in my real world problem the model is not just a linear layer. I guess with a detach solution I would need to redefine my whole model with functions. I guess wrapping with requires_grad is easier then. Is there a downside to using requires_grad compared to functions (aka the detach solution)?
I still think there could be some improvement of pytorch here. The difference you mentioned between no_grad and requires_grad is not clear and I think a specific parameter or environment for the “propagate gradient but don’t adjust weights” use-case would be helpful. It would make this much more explicit (I prefer explicitness).
What is the right place to propose such a feature request? Do you see any chance for such a feature request to succeed?