AndrewSoul:

Yeah, as you said, there isn’t any weight and bias parameters in features[30]. So I choose features[37], which is Conv2d layer in my model.


vgg16 does only have 30 layers on .features. Are you using vgg16_bn?



 AndrewSoul:

So I compared the gradient of ModelOutputs in layer[‘37’] and model.features[37].weight.grad.
The shapes are different, like I said in my last post:


Maybe, because there is still some mixup between model architectures?
As you can see in my dummy example, you’ll get the same gradients with the same shapes, if you use the same model architecture.
Could you make sure to compare exactly the same models?
Try to print the modules using print(model) and also compare the number of parameters.
If you have a shape mismatch in a certain layer, it means that the model architecture is different.