There is no batch_normalization. But, after block 3, 4 and 5, I am taking sigmoid of the feature maps and then multiplying the resultant with the feature maps.
attention = torch.mean(input_, dim=1, keepdim=True)
importance_map = torch.sigmoid(attention)
return input_.mul(importance_map)

could that be the reason?
Another question, why is self.features being updated three times in model2 but only once in model1? Is the flow of gradients not the same for both the models?