look at using the functional interface and registering your learnable parameters as nn.Parameter
You can do:
F.linear(self.weight) 

See such an example in this repo for: https://github.com/szagoruyko/diracnets/blob/master/diracconv.py#L38-L39 238