Solved by SimonW in post #6 
              
              
                A couple things: 

BN applies an affine transform. So you want to set the affine scaling weight to 1 before getting Y, i.e. B2.weight.data.fill_(1).
Use biased version of variance.
BN normalizes for data within each channel. So you should calculate Y's variance for each channel, rather than for all …