After spending a few hours on this, as a beginner in PyTorch, I will put the answer, perhaps it will be useful for some people out there:

For mean and variance we should use running_mean.data and running_variance.data when assigning the pre-trained weights. This is strange since for the other variables, assigning the weights directly without explicit .data
It is safer to use the variable.data to assign weights in general. The function to load weights would look like this:

def put_weights(batchnorm, conv, params_w, batch_norm=True):
	if batch_norm:
		bn_bs = params_w['beta']
		batchnorm.bias.data = torch.from_numpy(bn_bs)
		bn_ws = params_w['gamma']
		batchnorm.weight.data = torch.from_numpy(bn_ws)
		bn_mean = params_w['mean']
		batchnorm.running_mean.data = torch.from_numpy(bn_mean)
		bn_var = params_w['var']
		batchnorm.running_var.data = torch.from_numpy(bn_var)
	
	conv_bs = params_w['biases']
	conv.bias.data = torch.from_numpy(conv_bs)
	conv_ws = params_w['weights']
	conv.weight.data = torch.from_numpy(conv_ws).permute(3, 2, 0, 1)
	return batchnorm, conv