What do you mean? Why wouldn’t it? A linear layer is simply a projection of X dimensions to Y dimensions, e.g. 512 to 512, or, 768 to 1, or any other that you can think of. As you can see here:

  

      github.com
  

  
    huggingface/transformers/blob/04976a32dc555667afa994e8f918cbee88d84a4f/src/transformers/models/bert/modeling_bert.py#L1492 4



      
      
          def __init__(self, config):
              super().__init__(config)
              self.num_labels = config.num_labels
              self.config = config
          
          
    self.bert = BertModel(config)
              classifier_dropout = (
                  config.classifier_dropout if config.classifier_dropout is not None else config.hidden_dropout_prob
              )
              self.dropout = nn.Dropout(classifier_dropout)
              self.classifier = nn.Linear(config.hidden_size, config.num_labels)
          
          
    self.init_weights()
          
          
@add_start_docstrings_to_model_forward(BERT_INPUTS_DOCSTRING.format("batch_size, sequence_length"))
          @add_code_sample_docstrings(
              tokenizer_class=_TOKENIZER_FOR_DOC,
              checkpoint=_CHECKPOINT_FOR_DOC,
              output_type=SequenceClassifierOutput,
              config_class=_CONFIG_FOR_DOC,
          )
      
    
  


  

  
    
    
  

  


The linear layer will output the number of classes that you request. If you have five classes, it will output five values. How you deal with those values (single or multi label classification) then depends on your loss function. See:

  

      github.com
  

  
    huggingface/transformers/blob/04976a32dc555667afa994e8f918cbee88d84a4f/src/transformers/models/bert/modeling_bert.py#L1551-L1562 1



      
      
          if self.config.problem_type == "regression":
              loss_fct = MSELoss()
              if self.num_labels == 1:
                  loss = loss_fct(logits.squeeze(), labels.squeeze())
              else:
                  loss = loss_fct(logits, labels)
          elif self.config.problem_type == "single_label_classification":
              loss_fct = CrossEntropyLoss()
              loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
          elif self.config.problem_type == "multi_label_classification":
              loss_fct = BCEWithLogitsLoss()
              loss = loss_fct(logits, labels)