I would like to implement a custom convolution layer, where in I actually modify each individual part of the input with other external filter which also requires gradient.
I am succesful in implementing that in python, but it is very slow as expected.
Now to speed it up, I would like to implement the convolution in C++ front end. As shown here https://pytorch.org/tutorials/advanced/cpp_extension.html 12
But want to use the backward pass of the inbuilt convolution. Also the extra parameter which is a part of the layer should get the gradients as well.
Before starting it, I wanted to know whether I need to write my own backward pass, or does the c++ end take care of it.
Or is there any other simpler way, please suggest.
Thank you