Hi,
I’ve been trying to finetune the BART large pre-trained on MNLI with the Financial Phrasebank dataset to build a model for news sentiment analysis. I’m just a beginner and so, I mostly use the code from GEM Getting Started.
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, Seq2SeqTrainer, Seq2SeqTrainingArguments
tokenizer = AutoTokenizer.from_pretrained(‘facebook/bart-large-mnli’)
model = AutoModelForSeq2SeqLM.from_pretrained(‘facebook/bart-large-mnli’)
The model only trains when I use the AutoModelForSeq2SeqLM, Seq2SeqTrainer andSeq2SeqTrainingArguments. When I use model = AutoModelForSequenceClassification.from_pretrained(‘facebook/bart-large-mnli’) with the Trainer and TrainingArguments, the model does not train.
Is it appropriate to use seq2seq for sentiment classification tasks?
Any suggestions would be immensely helpful.
Thanks in advance.