Question on the example script run_glue.py for text classification
When we run this script to train a text classification, are the weights of the underlying language model frozen and not updated? Whether they are fixed or trainable, is there any config to change the behavior of the training process? Thanks!
hi @xiaolin-cheng 

`run_glue.py` fine-tunes the whole model, it doesn't freeze anything. You would need to manually freeze the base model, you could do this after loading the `ForSequenceClassification` and then freeze the base model. For example for `BertForSequenceClassification` you can access the base model using `model.bert`.

```python
model = BertForSequenceClassification.from_pretrained('bert-base-uncased')

for param in model.bert.parameters():
    param.requires_grad = False
```<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.