Ok, I’ve solved this.
So the basic solution is that, we need to regenerate the kernels every time after we update the sigma.  To update sigma, specify it in self._parameters. And in forward() function,  regenerate the kernels. Remember to do all the operations on GPU, otherwise the procedure may be slower.