I am doing Sentiment Analysis over some text reviews, but I do not get good results from.
I use BERT for feature extraction and a Fully Connected as classifier.
I am going to do these experiments, but I do not have any overview of the results in general. I have two options:
1- Unfreeze some Transfomer layers and let the gradient propagate over that layers
2- Do pre-train the BERT with masked language over related texts and then use classifier.
Which one has the priority? Or it depends just on experiments?