Thanks for the help these past few weeks. But I still have a doubt. I built Model A and Model B of MobileNet. Model A is build using Mask for pointwise convolutions of groups 2  which is Dense and Model B is build  using nn.Conv2d of groups 2. I tried to pass a random Tensor [1,3,32,32] to Model A and copied the weights of Model A to Model B [[1,0; 0, 1] --> [1;1] in this fashion. I am getting same output for both models. I loaded the trained weights and it is still giving the same error.
I thought validation accuracy should be same for both. but I am getting for Model A 85.9% and for Model B 10% on cifar-10 dataset.
When I disabled Net.eval(), I’m getting same Validation Accuracy for both Models 81.3%.
Case II:
I passed random Tensor as input to Models with ModelA.eval() and ModelB.eval() statements after loading weights from state_dict. I’m getting different outputs for both models compared to outputs for ModelA and ModelB which are same before initialising ModelA and ModelB.
What is the reason behind this? Can I still say ModelA and ModelB are behaving the same?
Update
Currently removing BatchNorm weights and training the model. Will validate the model and let you know. But I do not understand why BatchNorm should cause this error.