nielsr:

for name, param in model.named_parameters():
     if name.startswith("..."): # choose whatever you like here
        param.requires_grad = False



Thank you so much nielsr for the quick and useful reply. I believe I got this to work. So to verify, that can be written prior to “Trainer” command and will freeze any specified parameter? So for example, I could write the code below to freeze the first two layers.

for name, param in model.named_parameters():
if name.startswith(“bert.encoder.layer.1”):
param.requires_grad = False
if name.startswith(“bert.encoder.layer.2”):
param.requires_grad = False

This question shows my ignorance, but is there a way to print model settings prior to training to verify which layers/parameters are frozen?