Freezing base transformer weights
As I understand, say if I'm doing a classification task, then the transformer weights, along with the top classification layer weights, are both trainable (i.e. `requires_grad=True`), correct? If so, is there a way to freeze the transformer weights, but only train the top layer? Is that a good idea in general when I have a small dataset?
Hi!

You can modify the trainable attributes as described in #95.<|||||>Thanks!