In a simple test, using the original formula for InstanceNorm2d in here:

eps = 1e-5
N = nn.InstanceNorm2d(1, eps=eps)
x = torch.arange(25).view(1, 1, 5, 5).float()
y1 = N(x)
y2 = (x - x.mean())/torch.sqrt(x.var() + eps)

The error is quite big:

y2-y1
tensor([[[[ 0.0336,  0.0308,  0.0280,  0.0252,  0.0224],
[ 0.0196,  0.0168,  0.0140,  0.0112,  0.0084],
[ 0.0056,  0.0028,  0.0000, -0.0028, -0.0056],
[-0.0084, -0.0112, -0.0140, -0.0168, -0.0196],
[-0.0224, -0.0252, -0.0280, -0.0308, -0.0336]]]])

But the result is consistent. I get the same in CPU and GPU.
What is the right formula for InstanceNorm2d?