error when trying to use multilingual model for fine tuning
I wanted to use fine tuning for hindi language data. For that I tried to give bert-base-mutlilingual model but I am getting the following error

> python pregenerate_training_data.py --train_corpus=./hindi_pytorch_bert_data_1.txt --bert_model=bert-base-multilingual --output_dir=./hindi_train_data_1_3epochs/ --epochs_to_generate=3

```
Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex.
Model name 'bert-base-multilingual' was not found in model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese). We assumed 'bert-base-multilingual' was a path or url but couldn't find any file associated to this path or url.
Traceback (most recent call last):
  File "pregenerate_training_data.py", line 292, in <module>
    main()
  File "pregenerate_training_data.py", line 255, in main
    vocab_list = list(tokenizer.vocab.keys())
AttributeError: 'NoneType' object has no attribute 'vocab'
```

I tried giving bert-base-multilingual-cased as well then I ran into this error

> python pregenerate_training_data.py --train_corpus=./hindi_pytorch_bert_data_1.txt --bert_model=bert-base-multilingual-cased --output_dir=./hindi_train_data_1_3epochs/ --epochs_to_generate=3

```
Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex.
usage: pregenerate_training_data.py [-h] --train_corpus TRAIN_CORPUS
                                    --output_dir OUTPUT_DIR --bert_model
                                    {bert-base-uncased,bert-large-uncased,bert-base-cased,bert-base-multilingual,bert-base-chinese}
                                    [--do_lower_case] [--reduce_memory]
                                    [--epochs_to_generate EPOCHS_TO_GENERATE]
                                    [--max_seq_len MAX_SEQ_LEN]
                                    [--short_seq_prob SHORT_SEQ_PROB]
                                    [--masked_lm_prob MASKED_LM_PROB]
                                    [--max_predictions_per_seq MAX_PREDICTIONS_PER_SEQ]
pregenerate_training_data.py: error: argument --bert_model: invalid choice: 'bert-base-multilingual-cased' (choose from 'bert-base-uncased', 'bert-large-uncased', 'bert-base-cased', 'bert-base-multilingual', 'bert-base-chinese')
```

How to resolve this issue?
I made changes in the code pregenerate_training_data.py
from 
```
parser.add_argument("--bert_model", type=str, required=True,
                        choices=["bert-base-uncased", "bert-large-uncased", "bert-base-cased",
                                 "bert-base-multilingual", "bert-base-chinese"])
```
to
```
parser.add_argument("--bert_model", type=str, required=True,
                        choices=["bert-base-uncased", "bert-large-uncased", "bert-base-cased",
                                 "bert-base-multilingual-cased", "bert-base-multilingual-uncased", "bert-base-chinese"])
```


and it worked.<|||||>It occured to me maybe because I forgot to install pytorch. I installed pytorch then it's solved.<|||||>Hi, 
I followed your code, and got this error:

Traceback (most recent call last):      | 6796/185072 [00:00<00:18, 9787.42it/s]
  File "pregenerate_training_data.py", line 308, in <module>
    main()
  File "pregenerate_training_data.py", line 293, in main
    vocab_list=vocab_list)
  File "pregenerate_training_data.py", line 208, in create_instances_from_document
    assert len(tokens_b) >= 1
AssertionError

Can you please share your code?<|||||>What computer specification to train your corpus? How big it is and how long you need to training your corpus?

I wanna too train my corpus using fine tuning, maybe your answers give me an insight about how relevant me to training the corpus, thanks<|||||>> Hi,
> I followed your code, and got this error:
> 
> Traceback (most recent call last): | 6796/185072 [00:00<00:18, 9787.42it/s]
> File "pregenerate_training_data.py", line 308, in 
> main()
> File "pregenerate_training_data.py", line 293, in main
> vocab_list=vocab_list)
> File "pregenerate_training_data.py", line 208, in create_instances_from_document
> assert len(tokens_b) >= 1
> AssertionError
> 
> Can you please share your code?

I run into the same problem. Wondering if you have solved your problem. Thanks!<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
