Whom…
I don’t come up with any good idea at this moment, though, how about making NewNet handle PretrainedNet as its one layer like nn.Conv2d and nn.Linear?
Then it will be easier to load weights and change requires_grad. I’m not sure this is good but I paste a toy snippet below.
"""Definition"""
class PretrainedNet(nn.Module):

class NewNet(nn.Module):
    def __init__(self, **kwargs):
        super().__init__()
        self.pretrained = PretrainedNet()
        # register layers unique to NewNet below.
        ...

    def forward(self, x, **kwargs):
        # forward computation


new_net = NewNet()
pretrained_net_state_dict = torch.load(path/to/state_dict)  # or PretrainedNet()
new_net.pretrained.load_state_dict(pretrained_net_state_dict)
new_net.pretrained.requires_grad = False

Again, I’m not confident with if this works well for you. But I hope this helps.