Hello, thank you for your reply. I am using the pretrained version of the DenseNet from torchvision models. As it appears to me all the dense block attributes of the model are nn.Sequential and have BN and ReLU as modules in those sequential modules. Should I then rewrite the whole sequential module and replace the original one with that? The image may explain a little better what I am trying to say: