You could initialize the nn.Parameters in the model’s __init__ method, create the desired filter in the forward (e.g. via torch.cat and torch.stack) and apply it via the functional API (F.conv2d in your case).