Hmm that makes sense, but doesn’t help with my problem. I’m trying to do sentence(or rather clause) multiclass classification. I don’t want any signal from previous batch to current batch. However, I’m still facing the problem of small gradients, which leads to terrible accuracy and F1 score(the model just predicts the most common class in the data).