Solved by ptrblck in post #2 
              
              
                Yes, you could create a custom nn.Parameter for the trainable part and use torch.cat or torch.stack to create the weight tensor for the convolution. Once this is done, you could use the functional API via F.conv2d(input, weight, ...) to apply the convolution. 
You could use a custom nn.Module for th…