Hello everyone,
I have a pre-trained model called SoundNet, and the weights are available in TensorFlow. I created the model and I loaded the weights. The implementation is available here 8 .
The preprocessing and many parameters were taken from the Tensorflow repository
The output of the first convolutional layer is exactly the same as the one in Tensorflow, however, the following BatchNorm2d layer yields different numbers.

There is an ambiguity in the way we should use the mean and the variance of Batchnorm layer in Pytorch.  I am doubting about the way I load the weights of BatchNorm in Pytorch and the way I use it:

def put_weights(batchnorm, conv, params_w, batch_norm=True):
	if batch_norm:
		bn_bs = params_w['beta']
		batchnorm.bias = torch.nn.Parameter(torch.from_numpy(bn_bs))
		bn_ws = params_w['gamma']
		batchnorm.weight = torch.nn.Parameter(torch.from_numpy(bn_ws))
		bn_mean = params_w['mean']
		batchnorm.mean = torch.nn.Parameter(torch.from_numpy(bn_mean))
		bn_var = params_w['var']
		batchnorm.variance = torch.nn.Parameter(torch.from_numpy(bn_var))

	conv_bs = params_w['biases']
	conv.bias = torch.nn.Parameter(torch.from_numpy(conv_bs))
	conv_ws = params_w['weights']
	conv_ws = torch.from_numpy(conv_ws).permute(3, 2, 0, 1)
	conv.weight = torch.nn.Parameter(conv_ws)
	
	return batchnorm, conv

When I use the model to extract the features, I put the model in eval() mode
model.eval() # this to ensure that pre-calculated means and variances are used during feature extraction

Does anybody have an idea why the I got this kind of output? Do I use the model weights properly?