I noticed one way to solve this problem is to compute output=nn.functional.conv2d(input, weight) first, and then perform any custom computations to get an output_temp. Finally, replace the values of output with output_temp.
However, the drawback is that I have to compute convolution operations twice, which slows down the training a lot. Are there any other strategies available? Thanks!