No prediction for some words (BERT NER) when run on GPU
I have tried to make predictions over the test data using GPU, but ended up having no predictions for some words.  Any help would be appreciated.

Following is the shell script used for prediction.

export MAX_LENGTH=128
export BERT_MODEL=bert-base-multilingual-cased

DATA_DIR=../DATA/after_preprocess
OUTPUT_DIR=../CHECKPOINTS/GPU/CONLL_25L
LABELS_FILE_25=../DATA/after_preprocess/labels.txt

export BATCH_SIZE=32
export NUM_EPOCHS=3
export SAVE_STEPS=750
export SEED=1

cd ../examples

python3 -m torch.distributed.launch run_ner.py --data_dir $DATA_DIR/ \
--model_type bert \
--labels $LABELS_FILE_25 \
--model_name_or_path $BERT_MODEL \
--output_dir $OUTPUT_DIR \
--max_seq_length  $MAX_LENGTH \
--num_train_epochs $NUM_EPOCHS \
--per_gpu_train_batch_size $BATCH_SIZE \
--save_steps $SAVE_STEPS \
--seed $SEED \
--do_predict


Following is the error message:


02/17/2020 12:07:51 - INFO - transformers.tokenization_utils -   Model name '../CHECKPOINTS/GPU/CONLL_25L' not found in model shortcut name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc, bert-base-german-dbmdz-cased, bert-base-german-dbmdz-uncased, bert-base-finnish-cased-v1, bert-base-finnish-uncased-v1, bert-base-dutch-cased). Assuming '../CHECKPOINTS/GPU/CONLL_25L' is a path, a model identifier, or url to a directory containing tokenizer files.
02/17/2020 12:07:51 - INFO - transformers.tokenization_utils -   Didn't find file ../CHECKPOINTS/GPU/CONLL_25L/added_tokens.json. We won't load it.
02/17/2020 12:07:51 - INFO - transformers.tokenization_utils -   loading file ../CHECKPOINTS/GPU/CONLL_25L/vocab.txt
02/17/2020 12:07:51 - INFO - transformers.tokenization_utils -   loading file None
02/17/2020 12:07:51 - INFO - transformers.tokenization_utils -   loading file ../CHECKPOINTS/GPU/CONLL_25L/special_tokens_map.json
02/17/2020 12:07:51 - INFO - transformers.tokenization_utils -   loading file ../CHECKPOINTS/GPU/CONLL_25L/tokenizer_config.json
02/17/2020 12:07:51 - INFO - transformers.configuration_utils -   loading configuration file ../CHECKPOINTS/GPU/CONLL_25L/config.json
02/17/2020 12:07:51 - INFO - transformers.configuration_utils -   Model config BertConfig {
  "architectures": [
    "BertForTokenClassification"
  ],
  "attention_probs_dropout_prob": 0.1,
  "bos_token_id": 0,
  "directionality": "bidi",
  "do_sample": false,
  "eos_token_ids": 0,
  "finetuning_task": null,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "id2label": {
    "0": "LABEL_0",
    "1": "LABEL_1"
  },
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "is_decoder": false,
  "label2id": {
    "LABEL_0": 0,
    "LABEL_1": 1
  },
  "layer_norm_eps": 1e-12,
  "length_penalty": 1.0,
  "max_length": 20,
  "max_position_embeddings": 512,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_beams": 1,
  "num_hidden_layers": 12,
  "num_labels": 25,
  "num_return_sequences": 1,
  "output_attentions": false,
  "output_hidden_states": false,
  "output_past": true,
  "pad_token_id": 0,
  "pooler_fc_size": 768,
  "pooler_num_attention_heads": 12,
  "pooler_num_fc_layers": 3,
  "pooler_size_per_head": 128,
  "pooler_type": "first_token_transform",
  "pruned_heads": {},
  "repetition_penalty": 1.0,
  "temperature": 1.0,
  "top_k": 50,
  "top_p": 1.0,
  "torchscript": false,
  "type_vocab_size": 2,
  "use_bfloat16": false,
  "vocab_size": 119547
}

02/17/2020 12:07:51 - INFO - transformers.modeling_utils -   loading weights file ../CHECKPOINTS/GPU/CONLL_25L/pytorch_model.bin
02/17/2020 12:07:54 - INFO - __main__ -   Creating features from dataset file at ../DATA/after_preprocess/
02/17/2020 12:07:54 - INFO - utils_ner -   Writing example 0 of 5100
02/17/2020 12:07:54 - INFO - utils_ner -   *** Example ***
02/17/2020 12:07:54 - INFO - utils_ner -   guid: test-1
02/17/2020 12:07:54 - INFO - utils_ner -   tokens: [CLS] 1951 bis 1953 wurde der nÃ¶rdlich ##e Teil als Jugend ##burg des Ko ##lp ##ing ##werke ##s gebaut . [SEP]
02/17/2020 12:07:54 - INFO - utils_ner -   input_ids: 101 11200 10467 11087 10283 10118 28253 10112 13043 10223 32790 12248 10139 30186 35451 10230 32827 10107 25760 119 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   label_ids: -100 24 24 24 24 24 24 -100 24 24 24 -100 24 6 -100 -100 -100 -100 24 24 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100
02/17/2020 12:07:54 - INFO - utils_ner -   *** Example ***
02/17/2020 12:07:54 - INFO - utils_ner -   guid: test-2
02/17/2020 12:07:54 - INFO - utils_ner -   tokens: [CLS] Da Mu ##ck das Krieg ##ss ##chreiben nicht Ã¼ber ##bra ##cht hat , wird er als Re ##tter des Landes ausgezeichnet und soll zum Sc ##hat ##zm ##eister ernannt werden . [SEP]
02/17/2020 12:07:54 - INFO - utils_ner -   input_ids: 101 11818 49056 11263 10242 20587 13420 82089 10726 10848 13581 11640 11250 117 10790 10163 10223 20304 18413 10139 23244 32149 10130 17375 10580 55260 19180 37661 45940 27093 10615 119 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   label_ids: -100 24 9 -100 24 24 -100 -100 24 24 -100 -100 24 24 24 24 24 24 -100 24 24 24 24 24 24 24 -100 -100 -100 24 24 24 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100
02/17/2020 12:07:54 - INFO - utils_ner -   *** Example ***
02/17/2020 12:07:54 - INFO - utils_ner -   guid: test-3
02/17/2020 12:07:54 - INFO - utils_ner -   tokens: [CLS] Mit 1 . JÃ¤nner 2007 wurde Robert Sc ##h ##Ã¶r ##gen ##hof ##er , als Nachfolger des aus ##ges ##chie ##dene ##n Dietmar Dr ##abe ##k , in die Kader ##liste der FIFA - Sc ##hie ##ds ##richter aufgenommen . [SEP]
02/17/2020 12:07:54 - INFO - utils_ner -   input_ids: 101 12699 122 119 105531 10202 10283 10820 55260 10237 15020 11280 20202 10165 117 10223 27968 10139 10441 13156 50784 49906 10115 102411 11612 40929 10174 117 10106 10128 53361 26719 10118 13707 118 55260 72287 13268 59410 25919 119 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   label_ids: -100 24 24 -100 24 24 24 9 21 -100 -100 -100 -100 -100 24 24 24 24 24 -100 -100 -100 -100 9 21 -100 -100 24 24 24 24 -100 24 5 -100 -100 -100 -100 -100 24 24 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100
02/17/2020 12:07:54 - INFO - utils_ner -   *** Example ***
02/17/2020 12:07:54 - INFO - utils_ner -   guid: test-4
02/17/2020 12:07:54 - INFO - utils_ner -   tokens: [CLS] Die These , Sc ##hla ##tter sei Anti ##sem ##it gewesen , wurde seither in der theo ##logischen Fach ##lite ##ratu ##r nicht mehr vertreten . [SEP]
02/17/2020 12:07:54 - INFO - utils_ner -   input_ids: 101 10236 13252 117 55260 74935 18413 13868 26267 38443 10486 27044 117 10283 85983 10106 10118 13951 57325 100705 66289 50088 10129 10726 12471 41852 119 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   label_ids: -100 24 24 24 9 -100 -100 24 24 -100 -100 24 24 24 24 24 24 24 -100 24 -100 -100 -100 24 24 24 24 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100
02/17/2020 12:07:54 - INFO - utils_ner -   *** Example ***
02/17/2020 12:07:54 - INFO - utils_ner -   guid: test-5
02/17/2020 12:07:54 - INFO - utils_ner -   tokens: [CLS] " Le ##hm ##bru ##ck - Be ##uy ##s . Zeichnungen " lautet der Titel der gerade erÃ¶ffnete ##n Ausstellung , die Kur ##atori ##n Dr . Marion Born ##sche ##uer bis zum 11 . Januar im Le ##hm ##bru ##ck - Museum prÃ¤sentiert . [SEP]
02/17/2020 12:07:54 - INFO - utils_ner -   input_ids: 101 107 10281 29389 40309 11263 118 14321 53452 10107 119 96784 107 77566 10118 16076 10118 43234 61469 10115 41972 117 10128 61912 45804 10115 11612 119 27276 18021 12279 19047 10467 10580 10193 119 12468 10211 10281 29389 40309 11263 118 11325 91619 119 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
02/17/2020 12:07:54 - INFO - utils_ner -   label_ids: -100 24 6 -100 -100 -100 18 18 -100 -100 18 -100 24 24 24 24 24 24 24 -100 24 24 24 24 -100 -100 24 24 9 21 -100 -100 24 24 24 -100 24 24 3 -100 -100 -100 -100 -100 24 24 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100 -100
02/17/2020 12:07:57 - INFO - __main__ -   Saving features into cached file ../DATA/after_preprocess/cached_test_bert-base-multilingual-cased_128
02/17/2020 12:07:58 - INFO - __main__ -   ***** Running evaluation  *****
02/17/2020 12:07:58 - INFO - __main__ -     Num examples = 5100
02/17/2020 12:07:58 - INFO - __main__ -     Batch size = 8

Evaluating:   0%|          | 0/638 [00:00<?, ?it/s]
Evaluating: 100%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆ| 638/638 [00:27<00:00, 23.15it/s]


02/17/2020 12:08:27 - INFO - __main__ -   ***** Eval results  *****
02/17/2020 12:08:27 - INFO - __main__ -     f1 = 0.8600886024969794
02/17/2020 12:08:27 - INFO - __main__ -     loss = 0.070130527494103
02/17/2020 12:08:27 - INFO - __main__ -     precision = 0.8560205226871893
02/17/2020 12:08:27 - INFO - __main__ -     recall = 0.8641955325348009
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'wird'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'er'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'als'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Retter'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'des'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Landes'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'ausgezeichnet'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'und'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'soll'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'zum'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Schatzmeister'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'ernannt'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'werden'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for '.'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Nachfolger'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'des'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'ausgeschiedenen'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Dietmar'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Drabek'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for ','.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'in'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'die'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Kaderliste'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'der'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'FIFA-Schiedsrichter'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'aufgenommen'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for '.'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for '"'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'lautet'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'der'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Titel'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'der'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'gerade'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'erÃ¶ffneten'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Ausstellung'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for ','.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'die'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Kuratorin'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Dr'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for '.'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Marion'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Bornscheuer'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'bis'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'zum'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for '11.'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Januar'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'im'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Lehmbruck-Museum'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'prÃ¤sentiert'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for '.'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'an'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'der'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'SÃ¼dseite'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'des'.
02/17/2020 12:08:27 - WARNING - __main__ -   Maximum sequence length exceeded: No prediction for 'Saals'.

I tried  to make the predictions using CPU, and it worked just fine.  But the predictions made by CPU is totally different from the predictions made by GPU.  Isn't a model supposed to give the same predictions irrespective of whether it is loaded in GPU or CPU?

Any help would be appreciated.
Thanks in advance<|||||>Can you post a short reproducibility case?<|||||>@cibinjohn 

The error msg "Maximum sequence length exceeded" indicated that the input length (assume 257) was longer than the max_seq_len parameter (assume 256). In this case, the last token will not be predicted (actually be trimmed during the preprocessing). You have to reduce the length of your input or increase max_seq_len whichever works for you.

For the question of difference results between CPU and GPU, I cannot repeat a case similar to yours.<|||||>@cibinjohn 
I have been facing the same issue yesterday while testing on my own database. I tried doubling the max_seq_length value which was before equal to the MAX_LENGTH env variable. It worked for me.
<|||||>If any of you can post a short reproducible example, we can look into this.<|||||>@BramVanroy what I observed is that the prediction list has a list of different sizes which depends upon the length of the row data before every empty line in test.txt which is created after pre-processing using preprocess.py. 
So, I kept MAX_LENGTH as 128 and max_sequence length in the range of 190-256 and it worked.
According to me, it might be because of the length of the word_tokens list or maybe because of the length of the row data before every empty line of the test.txt file.
[test (5).txt](https://github.com/huggingface/transformers/files/4255447/test.5.txt)
If for this text file if you keep max_sequence_length as 128 then it will show Maximum_sequence_length_exceeded for around 400 tokens.<|||||> @BramVanroy 

%%capture
!pip install -qU transformers==2.4
!pip install -qU pytorch-lightning
!git clone --branch fixlight https://github.com/srush/transformers
!pip install -r transformers/examples/requirements.txt

%%capture
%%bash
cd transformers/examples/ner/
wget "https://raw.githubusercontent.com/stefan-it/fine-tuned-berts-seq/master/scripts/preprocess.py"
export MAX_LENGTH=128
export BERT_MODEL=bert-base-multilingual-cased
python3 preprocess.py train.txt.tmp $BERT_MODEL $MAX_LENGTH > train.txt
python3 preprocess.py dev.txt.tmp $BERT_MODEL $MAX_LENGTH > dev.txt
python3 preprocess.py test.txt.tmp $BERT_MODEL $MAX_LENGTH > test.txt
cat train.txt dev.txt test.txt | cut -d " " -f 2 | grep -v "^$"| sort | uniq > labels.txt


!cd transformers/examples/ner/; \
export MAX_LENGTH=190; \
export BERT_MODEL=bert-base-multilingual-cased; \
export OUTPUT_DIR=germeval-model; \
export BATCH_SIZE=32; \
export NUM_EPOCHS=3; \
export SAVE_STEPS=750; \
export SEED=42; \
python3 run_ner.py --data_dir ./ \
--model_type bert \
--labels ./labels.txt \
--model_name_or_path $BERT_MODEL \
--output_dir $OUTPUT_DIR \
--max_seq_length  $MAX_LENGTH \
--num_train_epochs $NUM_EPOCHS \
--per_gpu_train_batch_size $BATCH_SIZE \
--save_steps $SAVE_STEPS \
--seed $SEED \
--do_train \
--do_eval \
--do_predict<|||||>I met this question last day ，and i checked all cases but nothing has gone wrong。
so，i made a new dir，then just  put ‘train.txt’、“test.txt”、“dev.txt”、“labels.txt” there， start；

then，all is ok。
<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
