Based on your need.
If you want to keep some of the parameters fixed during training,
you can just ignore those parameters when you passing parameters to a optimizer:
trainable_parameters = []
for name, p in model.named_parameters():
    if "network2" not in name:
        trainable_parameters.append(p)

optimizer = torch.optim.SGD(params=trainable_parameters, lr=0.1, momentum=1e-5)