@sgugger
So I try to do some modification into https://github.com/huggingface/transformers/blob/master/examples/token-classification/run_tf_ner.py 30
I try to do what you said but I don’t know if I am doing it right or not. So what I am trying to do is to load the model as it is using from_pretrained and then change the last layer if the labels are not the same as the defined labels.
Can you please review it and told me if I am doing it right? Thank you in advance!
    config = AutoConfig.from_pretrained(
        model_args.config_name if model_args.config_name else model_args.model_name_or_path,
        num_labels=num_labels,
        id2label=label_map,
        label2id={label: i for i, label in enumerate(labels)},
        cache_dir=model_args.cache_dir,
    )
    tokenizer = AutoTokenizer.from_pretrained(
        model_args.tokenizer_name if model_args.tokenizer_name else model_args.model_name_or_path,
        cache_dir=model_args.cache_dir,
        use_fast=model_args.use_fast,
    )

    with training_args.strategy.scope():
        model = TFAutoModelForTokenClassification.from_pretrained(
            model_args.model_name_or_path,
            from_pt=bool(".bin" in model_args.model_name_or_path),
            cache_dir=model_args.cache_dir,
        )

        if model.config.num_labels != config.num_labels or model.config.id2label != config.id2label:
            model.classifier = tf.keras.layers.Dense(
                config.num_labels, kernel_initializer=get_initializer(config.initializer_range), name="classifier"
            )
        model.config = config