Hi,
I am new to Transformers/NLP.
I am trying to use Transformers for text classification.  If I am not classifying to one of the pre-made GLUE benchmarks (and using my own use-case classes & texts), do I have to “fine-tune” the model?  If I have 35k texts, and 2 labels (imbalanced – 98% vs 2%) can I just use the AutoModelforSequenceClassification to throw a softmax on the end of the transformer, and train that softmax?  Or do I fine-tune the whole thing, using this tutorial 9?
Thanks!  I am excited about better understanding the field and more effectively using the library.