In this case the weights are the learnable parameters for my ttqConv-Modules:


self.fpWeight = torch.nn.Parameter(torch.Tensor(inWCout, inWCin, inWH, inWH))


I tried to mimic the torch.nn.Conv2d interface/behaviour, i.e. the weights are “attributes” of these layer instances. In the ttqConv2d.forward() method, I pass these weights down to the nn.functional.Conv2d Function, which expects the weights explicitly as its function argument.
Maybe you can also write a wrapper module, which contains the weights as its parameters and then just calls customized functions during the forward pass? Than you could use the wrapper module in nn.Sequential as before?