In nn.Modules, non learnable parameters are registered as buffers. And this done automatically any time you do self.foo = my_tensor. Any learnable parameter should be an instance of a nn.Parameter (that is basically a Tensor that always require grad). So yes you can have non-learnable parameters easily. The thing is that the nn.Linear() layer is a learnable thing. If you go through the process of setting it’s weights by hand, you can also set the requires_grad to False in that process without any extra complexity.