I am using Hugging Face Models for NLP tasks.
I see a lot of online examples like below which freezes the bottom layers and only train only few top layers. Basic idea is to use transfer learning, without having to train model from scratch. Which makes sense to me.
for layer in model.layers[:-2]:
    layer.trainable = False

However i find that accuracy of my model improves significantly when train it from scratch, Are there any downsides to train transformer model from scratch? What is the right approach? When should you unfreeze or freeze layers? Please advice