Fine Tune Bert Ner using TFBertForTokenClassification.from_pretrained
Hey,

I am new to the transformers Bert Training world and am trying to fine tune Bert model for NER on a coNll like dataset. But its not training the model and giving the below error

ValueError: No gradients provided for any variable: ['tf_bert_for_token_classification_8/classifier/kernel:0', 'tf_bert_for_token_classification_8/classifier/bias:0'].


Below is my code

```py
tr_inputs = tf.convert_to_tensor(tr_inputs)
val_inputs = tf.convert_to_tensor(val_inputs)
tr_tags = tf.convert_to_tensor(tr_tags)
val_tags = tf.convert_to_tensor(val_tags)
tr_masks = tf.convert_to_tensor(tr_masks)
val_masks = tf.convert_to_tensor(val_masks)
tr_segs = tf.convert_to_tensor(tr_segs)
val_segs = tf.convert_to_tensor(val_segs)

input_features_dict = {"input_ids":tr_inputs, "attention_mask":tr_masks, "token_type_ids":tr_segs, 'labels':tr_tags}
val_features_dict = {"input_ids":val_inputs, "attention_mask":val_masks, "token_type_ids":val_segs, 'labels':tr_tags}
train_data = tf.data.Dataset.from_tensor_slices(input_features_dict)
batch_train_data = train_data.batch(batch_num)
valid_data = tf.data.Dataset.from_tensor_slices(val_features_dict)
batch_valid_data = valid_data.batch(batch_num)
modell = TFBertForTokenClassification.from_pretrained('bert-base-uncased',num_labels=len(tag2idx))
modell.layers[2].activation = tf.keras.activations.softmax
modell.layers[0].trainable = False
modell.compile(optimizer=optimizer, loss=loss, metrics=[metrics])
modell.fit(batch_train_data, epochs=epochs, validation_data=batch_val_data)
```

Not sure what needs to be done. Any advice/pointers on this would be highly helpful for me.
What is your `metrics`?<|||||>```py
optimizer= AdamWeightDecay(
    learning_rate=5e-5,
    beta_1=0.9,
    beta_2=0.999,
    weight_decay_rate=0.01,
    epsilon=1e-6,
    exclude_from_weight_decay=['layer_norm', 'bias'])
optimizer._HAS_AGGREGATE_GRAD = False
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metrics=[tf.keras.metrics.SparseCategoricalAccuracy(name="acc")]
```<|||||>@jplu might know what's going on<|||||>Hello @aks2193!

Sorry for this, but for now you cannot use `.compile()` + `.fit()` to train a Token Classification model. To make it short, this is because a layer is not used and then the gradients will be None, something that `.fit()` cannot handle.

If you want to train a NER I suggest you to use the [example](https://github.com/huggingface/transformers/blob/master/examples/token-classification/run_tf_ner.py).<|||||>Hey @jplu 
Thanks for the reply. I tried to follow the link and below is how I changed my code

modell = TFBertForTokenClassification.from_pretrained('bert-base-uncased',num_labels=len(tag2idx))
modell.layers[2].activation = tf.keras.activations.softmax
modell.layers[0].trainable = False
modell.compile(optimizer=optimizer, loss=loss, metrics=[metrics])
modell.fit(batch_train_data, epochs=epochs, validation_data=batch_val_data)

def compute_metrics(pred):
        labels = pred.label_ids
        preds = pred.predictions.argmax(-1)
        precision, recall, f1, _ = precision_recall_fscore_support(labels, preds, average='binary')
        acc = accuracy_score(labels, preds)
        return {
            'accuracy': acc,
            'f1': f1,
            'precision': precision,
            'recall': recall
        }
training_args = TFTrainingArguments(
        output_dir='./bert_test',          # output directory
        num_train_epochs=5,              # total # of training epochs
        per_device_train_batch_size=32,  # batch size per device during training
        per_device_eval_batch_size=32,   # batch size for evaluation
        warmup_steps=500,                # number of warmup steps for learning rate scheduler
        weight_decay=0.01,               # strength of weight decay
        logging_dir='./logs',            # directory for storing logs
        learning_rate=3e-5,
    )

trainer = TFTrainer(
        model = modell,
        args=training_args,
        train_dataset=train_dataset,
        eval_dataset=val_dataset,
        compute_metrics=compute_metrics
    )
    
trainer.train()

But I am still getting the below error
 ValueError: Trying to create optimizer slot variable under the scope for tf.distribute.Strategy (<tensorflow.python.distribute.one_device_strategy.OneDeviceStrategy object at 0x7fb46c2dec50>), which is different from the scope used for the original variable (<tf.Variable 'tf_bert_for_token_classification_14/classifier/kernel:0' shape=(768, 10) dtype=float32
Make sure the slot variables are created under the same strategy scope. This may happen if you're restoring from a checkpoint outside the scope


Then I tried defining a strategy scope and include all the above code inside that

strategy = tf.distribute.OneDeviceStrategy(device="/cpu:0")
with strategy.scope():
    The ABOVE CODE

On doing this getting the below error

Mixing different tf.distribute.Strategy objects: <tensorflow.python.distribute.one_device_strategy.OneDeviceStrategy object at 0x7fb446d98eb8> is not <tensorflow.python.distribute.one_device_strategy.OneDeviceStrategy object at 0x7fb45fe2d7b8>

How does this exactly work?
How do I define the strategy scope for all this calculations?<|||||>Hey @aks2193 
I'm facing the same problem. Please let me know if you find a solution.<|||||>@aks2193 
Try replacing `strategy = tf.distribute.OneDeviceStrategy(device="/cpu:0")` to `training_args.strategy.scope()`. Worked for me.<|||||>@alibi123 has right, you are not properly instanciate your model. Please use the example as it is. You example won't work as well with the `TFTrainer` if you are setting the activation to `softmax` because we don't compute the loss from the logits.<|||||>@jplu I have a question about restoring weights from the checkpoint. How to do it correctly?

This is how I try to load weights:
```
>>> model = TFBertForTokenClassification.from_pretrained(settings.BERT_NAME)
>>> model.load_weights('/models/exp1/checkpoint/ckpt-55')
```
I get very long exception message starting with:
```
Nothing except the root object matched a checkpointed value. Typically this means that the checkpoint does not match the Python program. The following objects have no matching checkpointed value: [<tf.Variable 't
```

Here's my training code:
```
    dataset = get_dataset(in_fn, debug)
    args = TFTrainingArguments(
        os.path.join(os.path.join(settings.MODELS_DIR, save_name)),
        overwrite_output_dir=True,
        do_train=True,
        logging_dir=os.path.join(settings.DATA_DIR, 'exp1_logs'),
        save_total_limit=2,
    )
    with args.strategy.scope():
        model = TFBertForTokenClassification.from_pretrained(settings.BERT_NAME)
    trainer = TFTrainer(
        model=model,
        args=args,
        train_dataset=dataset,
    )
    trainer.train()
```
`BERT_NAME = 'bert-base-multilingual-cased'`

I was also trying to use ckpt path in `.from_pretrained()` but also got errors regarding format.
<|||||>You cannot use Keras `load_weights` on a TF checkpoint. If you want to load your model you just have to use the path where you saved your model `model = TFBertForTokenClassification.from_pretrained("my_output_dir")`<|||||>@jplu Thank you for the quick response.
I've tried that in the first place, but get an error as well. 

This is my output dir:
```
checkpoint$ ls
checkpoint  ckpt-55.data-00000-of-00001  ckpt-55.index  ckpt-56.data-00000-of-00001  ckpt-56.index
```
Here are my attempts with error messages:
1:
```
config = BertConfig.from_pretrained(settings.BERT_NAME)
>>> model = TFBertForTokenClassification.from_pretrained('/models/exp1/checkpoint', config=config)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 653, in from_pretrained
    [WEIGHTS_NAME, TF2_WEIGHTS_NAME], pretrained_model_name_or_path
OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5'] found in directory /models/exp1/checkpoint or `from_pt` set to False
```
2:
```
>>> model = TFBertForTokenClassification.from_pretrained('/models/exp1/checkpoint/ckpt-55', config=config)
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 711, in from_pretrained
    load_tf_weights(model, resolved_archive_file)
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 268, in load_tf_weights
    with h5py.File(resolved_archive_file, "r") as f:
  File "/usr/local/lib/python3.6/dist-packages/h5py/_hl/files.py", line 408, in __init__
    swmr=swmr)
  File "/usr/local/lib/python3.6/dist-packages/h5py/_hl/files.py", line 173, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 88, in h5py.h5f.open
OSError: Unable to open file (file signature not found)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 714, in from_pretrained
    "Unable to load weights from h5 file. "
OSError: Unable to load weights from h5 file. If you tried to load a TF 2.0 model from a PyTorch checkpoint, please set from_pt=True.
```
3: I've even tried `from_pt=True` even though I used TFTrainer and TFBert
```
>>> model = TFBertForTokenClassification.from_pretrained('/models/exp1/checkpoint/ckpt-55', config=config, from_pt=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 703, in from_pretrained
    return load_pytorch_checkpoint_in_tf2_model(model, resolved_archive_file, allow_missing_keys=True)
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_pytorch_utils.py", line 89, in load_pytorch_checkpoint_in_tf2_model
    pt_state_dict = torch.load(pt_path, map_location="cpu")
  File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 595, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 764, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '\x00'.
```
4: I've also tried to add config.json into the output dir
```
>>> model = TFBertForTokenClassification.from_pretrained('/models/exp1/checkpoint')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 653, in from_pretrained
    [WEIGHTS_NAME, TF2_WEIGHTS_NAME], pretrained_model_name_or_path
OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5'] found in directory /models/exp1/checkpoint or `from_pt` set to False
```<|||||>As I said, these are normal errors because: **You cannot use Keras load_weights on a TF checkpoint.** You have to use your output dir not the file or the checkpoint dir: `model = TFBertForTokenClassification.from_pretrained('/models/exp1')`.<|||||>@jplu Sorry for bothering. But still doesn't work. It expects `'pytorch_model.bin', 'tf_model.h5'`.
```
>>> model = TFBertForTokenClassification.from_pretrained('/models/exp1')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 653, in from_pretrained
    [WEIGHTS_NAME, TF2_WEIGHTS_NAME], pretrained_model_name_or_path
OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5'] found in directory /models/exp1 or `from_pt` set to False

>>> model = TFBertForTokenClassification.from_pretrained('/models/exp1', from_pt=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 653, in from_pretrained
    [WEIGHTS_NAME, TF2_WEIGHTS_NAME], pretrained_model_name_or_path
OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5'] found in directory /models/exp1 or `from_pt` set to False

```

But my output_dir only contains `checkpoint` dir<|||||>This is because you are trying to load a PyTorch model into a TensorFlow one with `from_pt=True`, remove this parameter. If not working it means that your models have not been properly saved.

Did you call the `save` method of the trainer?<|||||>No, I haven't. Sorry, my bad. I thought that I can use checkpoints. 
Thanks for your help!<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
