Abirate:

For T5, about 3 approaches have been used to fine-tune the pre-trained model for all the downstream tasks (1- fine-tuning all the pretrained layers (all params), 2- freeze all the pre-trained layers and adding adapter layers, and 3- gradual unfreezing the pre-trained layers over time (and it is clear in the paper (the tables showing GLUE for each of them). Ma question is about BERT, for BERT which of these approaches was used for fine-tuning it?


This question is answered at 1:10:35 in the main stream