The problem I’m facing is that I want to insert a small pre-trained model to an existing model to do something like features enhancement.
Whereas I want to know if the freezing operation (setting the requires_grad flag of parameters to False) will influence the gradient calculation especially for the layers before the inserted block.
def __init__(self, block, num_blocks, num_classes=10):
    ...
    ce = MGN_FIN()
   
    # load CE parameters
    ce = torch.nn.DataParallel(ce)
    ckp = torch.load('...')
    ce.load_state_dict(ckp['model'])

    # freeze CE layer's weights
    for name, p in ce.module.named_parameters():
        p.requires_grad = False

    # construct CE module
    self.CE = nn.Sequential(
        ce.module.block1,
        ce.module.block2,
        ce.module.block3,
    )  

...
def forward(self, x):
    out = F.relu(self.bn1(self.conv1(x)))
    out = self.CE(out)

    out = self.layer1(out)
    out = self.CE(out)

    out = self.layer2(out)
    out = self.layer3(out)
    out = F.avg_pool2d(out, out.size()[3])
    out = out.view(out.size(0), -1)
    out = self.linear(out)
    return out


For example:

Any help is appreciated!