Thanks for the code. You are using inplace operations so I would expect to see different results between both approaches, since the model would directly manipulate the batchnorm outputs via nn.Hardtanh e.g. in:
            nn.BatchNorm2d(128*self.infl_ratio),
            nn.Hardtanh(inplace=True),

A quick check after disabling the inplace behavior and using forward hooks yields the same results:
model = VGG_Cifar10()
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=1.)
model.eval()

import torch
images = torch.randn(1, 3, 224, 224)
act = []
model.features[2].register_forward_hook(lambda m, input, output: act.append((input, output)))
with torch.no_grad():
    output = model(images)

input = act[0][0][0]
output = act[0][1]
m = model.features[2]
out = F.batch_norm(input, weight=m.weight, bias=m.bias, running_mean=m.running_mean, running_var=m.running_var, training=m.training, eps=m.eps)
print((out - output).abs().max())
> tensor(0., grad_fn=<MaxBackward1>)