As you can see, the classifier is a single dense layer.
It is probably pointing out from here if you are using BertForSequenceClassification, transformers/modeling_bert.py at b29eb247d39b56d903ea36c4f6c272a7bb0c0b4c · huggingface/transformers · GitHub 14
If you are using BertForTokenClassification, it is pointing out here,

  
      github.com
  
  
    huggingface/transformers/blob/b29eb247d39b56d903ea36c4f6c272a7bb0c0b4c/src/transformers/models/bert/modeling_bert.py#L1648 16

class BertForTokenClassification(BertPreTrainedModel):


    _keys_to_ignore_on_load_unexpected = [r"pooler"]


    def __init__(self, config):
        super().__init__(config)
        self.num_labels = config.num_labels


        self.bert = BertModel(config, add_pooling_layer=False)
        self.dropout = nn.Dropout(config.hidden_dropout_prob)
        self.classifier = nn.Linear(config.hidden_size, config.num_labels)


        self.init_weights()


    @add_start_docstrings_to_model_forward(BERT_INPUTS_DOCSTRING.format("batch_size, sequence_length"))
    @add_code_sample_docstrings(
        tokenizer_class=_TOKENIZER_FOR_DOC,
        checkpoint=_CHECKPOINT_FOR_DOC,
        output_type=TokenClassifierOutput,
        config_class=_CONFIG_FOR_DOC,
    )



  
  
    
    
  
  


For setting up num_of_label, please change the variable in your config file.