You mean something like:



 Mirsadeghi:

x = extra_BatchNorm(x)*base_BatchNorm.weight+base_BatchNorm.bias


My issue is that, I want to apply this operation to other deep nets (resnet) which I can not directly change their forward pass methods as there are multiple use of bachNorm. Besides, I have to do this for every net and it is not possible. Can I apply this call:



 Mirsadeghi:

x = extra_BatchNorm(x)*base_BatchNorm.weight+base_BatchNorm.bias


for bachnorm layers without changing their code internally. Something like ‘model.apply’?