I am using a VGG19 network to classify CIFAR10 images, yet I have slightly different results for data of different batch sizes. Code:

f_image = net.forward(Variable(image, requires_grad=True)).data
f_image2 = net.forward(Variable(image[0].unsqueeze(0), requires_grad=True)).data
f_image3 = net.forward(Variable(image[0].unsqueeze(0), requires_grad=True)).data
print(’-----------------------------------------’)
print(‘f_image - f_image2 = {}’.format((f_image[0] - f_image2).cpu().data.numpy()))
print(‘f_image - f_image3 = {}’.format((f_image[0] - f_image3).cpu().data.numpy()))
print(‘f_image2 - f_image3 = {}’.format((f_image2 - f_image3).cpu().data.numpy()))
print(’-----------------------------------------’)

Specifically, image is of size [batch_size, n_channels, height, width]. And I expect f_image[0].unsqueeze(0) to be the same as f_image2 and f_image3. However, they are slightly different:


f_image - f_image2 = [[ 0.0000000e+00  3.7252903e-09 -1.8626451e-09  3.7252903e-09
0.0000000e+00 -3.7252903e-09  3.7252903e-09 -1.8626451e-09
3.7252903e-09 -1.3969839e-09]]
f_image - f_image3 = [[ 0.0000000e+00  3.7252903e-09 -1.8626451e-09  3.7252903e-09
0.0000000e+00 -3.7252903e-09  3.7252903e-09 -1.8626451e-09
3.7252903e-09 -1.3969839e-09]]
f_image2 - f_image3 = [[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]
-----------------------------------------]

The difference is very small, is it a bug or expected behavior of PyTorch?