Just in case you want to check
Here is 1 the data over which I want to perform batch norm, here is the 2d batch norm layer I am using and here is 1 the 3d batchnorm layer I am using
bn2d = torch.load('bn2d.pth', map_location=lambda storage, loc: storage)
bn3d = torch.load('bn3d.pth', map_location=lambda storage, loc: storage)
data = torch.load('2D-data.pth', map_location=lambda storage, loc: storage)

> o2 = bn2d(data)
> o2
# Prints long tensor

> o3 = bn3d(data[:,:,None,:,:].expand(1,64,32,128,128))[:,:,0,:,:]
> o3
# Prints another long tensor similar to dimension of above one

> o2.norm()
# tensor(156.6110)
> o3.norm()
# tensor(157.0714)

> (o3*o2).sum()/(o2.norm() * o3.norm())
#tensor(0.9985)

0.9985 is angle between the to vectors (matrices) in matrix space. It’s not very bad but yeah would love if it gets exact 1!!
I am debugging a network based on static image video so really need it!!