I have the same question about the batchnorm , I also need to extracte the parameters in the batchnorm. But i looked at the method you wrote MyBatchNorm, the final calculation of x.dta is incomplete, if compared with this batchnorm caluation   y = \frac{x - mean[x]}{ \sqrt{Var[x] + \epsilon}} * gamma + beta , the lack of part of this  {- mean[x] * gamma}{\sqrt{Var[x] + \epsilon}}. I added the program as shown in the figure below, while the program runs without error, but the cause of network loss is very serious, and so i think it is wrong to modify if, to ask you how to rewrite this part of the expression of the code, for batchnorm1d and batchnorm2d, should respectively how to write, I hope you can give answer, thank you.

1.PNG1283×483 14.6 KB