New tokenizers issue in NER demo
# 🐛 Bug

When I use transformers 2.5 I get the following error when running the run_ner demo. It seems to work from when I use 2.4. I am guessing this is because of a slight difference in the tokenization script? It seems to fail even when running with the base German run.sh in the ner/ directory. 

```Traceback (most recent call last):
  File "run_pl_ner.py", line 233, in <module>
    trainer = generic_train(model, args)
  File "/content/transformers/examples/ner/transformer_base.py", line 268, in generic_train
    trainer.fit(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/trainer.py", line 911, in fit
    self.single_gpu_train(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/distrib_parts.py", line 464, in single_gpu_train
    self.optimizers, self.lr_schedulers = self.init_optimizers(model.configure_optimizers())
  File "/content/transformers/examples/ner/transformer_base.py", line 92, in configure_optimizers
    * float(self.hparams.num_train_epochs)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/core/decorators.py", line 19, in _get_data_loader
    value = fn(self)  # Lazy evaluation, done only once.
  File "/content/transformers/examples/ner/transformer_base.py", line 132, in train_dataloader
    return self.load_dataset("train", self.hparams.train_batch_size)
  File "run_pl_ner.py", line 50, in load_dataset
    dataset = self.load_and_cache_examples(labels, self.pad_token_label_id, mode)
  File "run_pl_ner.py", line 175, in load_and_cache_examples
    pad_token_label_id=pad_token_label_id,
  File "/content/transformers/examples/ner/utils_ner.py", line 182, in convert_examples_to_features
    assert len(label_ids) == max_seq_length
```
This might be https://github.com/huggingface/transformers/issues/2917<|||||>@srush I'm looking at it right now <|||||>I have the same issue while running run_tf_ner.py on the German NER dataset. I got the same AssertionError as below:
>  File "run_tf_ner.py", line 655, in <module>
>     app.run(main)
>   File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 299, in run
>     _run_main(main, args)
>   File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 250, in _run_main
>     sys.exit(main(argv))
>   File "run_tf_ner.py", line 540, in main
>     args, tokenizer, labels, pad_token_label_id, train_batch_size, mode="train"
>   File "run_tf_ner.py", line 451, in load_and_cache_examples
>     pad_token_label_id=pad_token_label_id,
>   File "/content/transformers/examples/ner/utils_ner.py", line 182, in convert_examples_to_features
>     assert len(label_ids) == max_seq_length
> AssertionError

My idea is that the [pad_token_label_id = 0](https://github.com/huggingface/transformers/blob/94ff2d6ee8280c5595b92c1128c0f18e44925e56/examples/ner/run_tf_ner.py#L511)  may conflict with the orginal label_list id. Becase in utils_ner.py (line 104), `label_map = {label: i for i, label in enumerate(label_list)}` . 
By the way, I run the same code on CoNLL-2003 dataset with default labels:
["O", "B-MISC", "I-MISC", "B-PER", "I-PER", "B-ORG", "I-ORG", "B-LOC", "I-LOC"]
No such error message... It may be 'O' is the first token but in Germen dataset 'O' was the last label in label.text.  
I don't know if this is the real problem, but I hope this bug can be fixed soon. Thx.
 
<|||||>> @srush I'm looking at it right now

I have the same issue as @srush. Any idea yet what the issue could be?<|||||>It works fine with 2.4 so it is likely an issue with the new tokenizer.<|||||>Hi @srush ,

I am using transformers==2.4.1, but still facing the problem with a custom data set (with extra labels).  It is working with the german dataset though.  Could you be more specific about the version of transformers that you use.
Thanks
<|||||>Hi @cibinjohn , 

Can you tell us which tokenizers / model you're using ? We fixed something for `bert-base-multilingual` should be merge in master quite soon.
<|||||>Any one got solution for this yet? I have the same issue with this. I used transformer 2.5.1 and the latest tokenizer come with the transformers installation by default. 
Looks like the assert len(label_ids) == max_seq_length because len(label_ids) is one more than max_seq_length while 3 other asserts before it pass the tests. <|||||>@yuyongze  Have you made any progress on this?

I think `pad_token_label_id = 0` is actually another issue: #3332 
And related: https://stackoverflow.com/questions/60732509/label-handling-confusion-in-run-tf-ner-example<|||||>Experiencing the same problem launching run_ner.py on the WNUT17 dataset, although on German and CONLL-2003 everything works fine.<|||||>I just want to run the ner demo (TF-version) but the same issue/ error raises.. Tried with transformers version 2.4/5/6 still the same error raises. Has anyone a solution? 

Edit: PyTorch script seems to work<|||||>@mfuntowicz I am using `bert-base-multilingual` model & transformers==2.4.1.  

Could you let me know when the solution will be merged with master.  Looking forward to hear from you.
Thanks in advance

<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
