Hey! Just wanted to state that I am facing the same problem and still don’t know how this is happening. I was working with RoBERTaForSequenceClassification with a binary classification problem and it gave 50% ACC. after 5 epochs. It did not learn anything at all.