I’ve been unsuccessful in freezing lower pretrained BERT layers when training a classifier using Huggingface. I’m using AutoModelForSequenceClassification particularly, via code below, and I want to freeze the lower X layers (ex: lower 9 layers). Is this possible in HuggingFace, and if so what code would I add to this for functionality?

tokenizer = AutoTokenizer.from_pretrained(“bert-base-cased”)
def tokenize_function(examples):
return tokenizer(examples[“text”], max_length = 512, padding=“max_length”, truncation=True)
tokenized_train = train.map (tokenize_function, batched=True)
tokenized_test= test.map (tokenize_function, batched=True)
model = AutoModelForSequenceClassification.from_pretrained(“bert-base-cased”, num_labels=1)
for w in model.bert.parameters():
w._trainable= False
training_args = TrainingArguments(“test_trainer”, evaluation_strategy=“epoch”, per_device_train_batch_size=8)
trainer = Trainer(model=model, args=training_args, train_dataset=tokenized_train, eval_dataset=tokenized_test)
trainer.train()