From looking at BertForSequenceClassification it seems that you need

  

      github.com
  

  
    huggingface/transformers/blob/c02cd95c56249e9bd38ecb3e4ebcce6d9eebd4a4/src/transformers/models/bert/modeling_bert.py#L1558-L1559 3



      
      
          loss_fct = CrossEntropyLoss()
          loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
      
    
  


  

  
    
    
  

  


It has been a while since I used weighted CE so make sure to check the documentation that your way of using weights is correct. It always confuses me.