The exact implementation of XXXForSequenceClassification differs from model to model. But you can have a look at e.g. the BERT implementation:

  

      github.com
  

  
    huggingface/transformers/blob/04976a32dc555667afa994e8f918cbee88d84a4f/src/transformers/models/bert/modeling_bert.py#L1481 25



      
      
                  )
          
          

          
@add_start_docstrings(
              """
              Bert Model transformer with a sequence classification/regression head on top (a linear layer on top of the pooled
              output) e.g. for GLUE tasks.
              """,
              BERT_START_DOCSTRING,
          )
          class BertForSequenceClassification(BertPreTrainedModel):
              def __init__(self, config):
                  super().__init__(config)
                  self.num_labels = config.num_labels
                  self.config = config
          
          
        self.bert = BertModel(config)
                  classifier_dropout = (
                      config.classifier_dropout if config.classifier_dropout is not None else config.hidden_dropout_prob
                  )
                  self.dropout = nn.Dropout(classifier_dropout)
      
    
  


  

  
    
    
  

  


In this case it is simply a dropout and then a linear layer on top of the pooled outputs.