You freeze the parameters manually before training. For example, if i want to freeze the first 3 layers of a Resnet encoder, I use–
for i,child in enumerate(model.children()):
   for k,child_0 in enumerate(child.children()):
      if k<=3:
         for params in child_0.parameters():
            params.requires_grad=False
               print("Frozen {} layer {}".format(k,child_0))
   break

This is a small snippet which i built based on my encoder. You have to come up with your own freezing loop to freeze custom layers. Note that all this happens before training begins.
A very good idea would be to put it just after you have defined the model. After this, you define the optimizer as
optim = torch.optim.SGD(filter(lambda p: p.requires_grad, model.parameters()), lr, momentum=momentum, weight_decay=decay, nesterov=True) 

and you are good to go ! You can use this model in the training loop and the frozen layers will not have their weights updated.