You will have to do that manually. The Trainer won’t do it automatically for you. To freeze the pytorch model, you can do something like this:
model = AutoModel.from_pretrained(model_id)
for param in model.parameters():
    param.requires_grad = False

In this way, you’d have more granular control over which layers you want to freeze. For your use case, you’d want to train just the linear layer initially. You can do this by freezing just the encoder
for param in model.encoder.parameters():
    param.requires_grad = False

You’d have to check if encoder is an attribute of all models to be fully sure. At later stages, you can turn the bool to True, to train the encoder.
I hope this gives you an idea.