tensor size mismatch in NER.py
Hi,

I am trying to classify tokens for a task similar to NER. I am using the following code:

transformers/examples/pytorch/token-classification/run_ner.py
 
My input data is in JSON format for both train and test. When I perform classification using BioBERT, it runs without any issue. When I use BERT base (cased or uncased), I get the following error in the middle of the training:

Traceback (most recent call last):
  File "/home/tv349/PharmaBERT/transformers/examples/pytorch/token-classification/run_ner.py", line 530, in <module>
    main()
  File "/home/tv349/PharmaBERT/transformers/examples/pytorch/token-classification/run_ner.py", line 463, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/home/tv349/.conda/envs/HF/lib/python3.8/site-packages/transformers/trainer.py", line 1269, in train
    tr_loss += self.training_step(model, inputs)
  File "/home/tv349/.conda/envs/HF/lib/python3.8/site-packages/transformers/trainer.py", line 1754, in training_step
    loss = self.compute_loss(model, inputs)
  File "/home/tv349/.conda/envs/HF/lib/python3.8/site-packages/transformers/trainer.py", line 1786, in compute_loss
    outputs = model(**inputs)
  File "/home/tv349/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/tv349/.conda/envs/HF/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py", line 1712, in forward
    outputs = self.bert(
  File "/home/tv349/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/tv349/.conda/envs/HF/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py", line 984, in forward
    embedding_output = self.embeddings(
  File "/home/tv349/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/tv349/.conda/envs/HF/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py", line 221, in forward
    embeddings += position_embeddings
RuntimeError: The size of tensor a (607) must match the size of tensor b (512) at non-singleton dimension 1

Also, this is how I call the function:

python transformers/examples/pytorch/token-classification/run_ner.py \
  --model_name_or_path $model_path \
  --train_file $train_file \
  --validation_file $validation_file \
  --output_dir $root_output_dir \
  --overwrite_output_dir \
  --do_train \
  --do_eval

I don't think the problem is in my input data or setting because everything works for BioBERT, but failed for BERT base.

Could you please let me know what can I do to avoid this issue?

Thanks!
cc @sgugger <|||||>Mmm the problem looks like it's coming from the position embedding with an input of size 607, which is larger than 512 (the maximum handled apparently).<|||||>Thanks for the response. But why do you think it works for BioBert and not BERT base?<|||||>Will it help if I provide my training data?<|||||>Anything that could helps us reproduce the issue would help, yes.<|||||>> But why do you think it works for BioBert and not BERT base?

It seems this is happening due to the [`model_max_length`] property in `tokenizer_config.json`. So far I could understand from the [docs], setting this property to a value will trim the tokens while tokenization (please, correct me if I am wrong).

[`tokenizer_config.json` of biobert] sets `model_max_length=512`. But in case of bert-base [cased] or [uncased], `model_max_length` is not specified.

[`model_max_length`]: https://huggingface.co/transformers/internal/tokenization_utils.html?highlight=model_max_length#transformers.tokenization_utils_base.PreTrainedTokenizerBase
[docs]: https://huggingface.co/transformers/internal/tokenization_utils.html?highlight=model_max_length#transformers.tokenization_utils_base.PreTrainedTokenizerBase
[`tokenizer_config.json` of biobert]: https://huggingface.co/dmis-lab/biobert-v1.1/blob/main/tokenizer_config.json
[cased]: https://huggingface.co/bert-base-cased/blob/main/tokenizer_config.json
[uncased]: https://huggingface.co/bert-base-uncased/blob/main/tokenizer_config.json<|||||>Yes, that is very likely the case. I think the script is simply missing a `max_seq_length` parameter (like there is for text classification for instance) that the user should set when the `model_max_length` is either too large or simply missing.

I will add that tomorrow.<|||||>Adding model_max_length=512 to the tokenizer_config.json solved the issue. Thank you very much!<|||||>@sgugger that would be a great idea. I was looking for that option but couldn't find it. Thank you very much for your consideration.