UnboundLocalError: local variable 'i' referenced before assignment when using fine_tuning code
Hi @thomwolf 
I am using the lm_finetuning codes. Generated training data using generate_pretraining_data.py

When running finetune_on_pregenerated.py . I am getting this error.

logs

 python finetune_on_pregenerated.py --pregenerated_data training_1/ --bert_model bert-base-uncased --do_lower_case --output_dir finetuned_lm_1Msents_3epochs/ --epochs 3
Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex.
2019-04-13 09:32:43,014: device: cuda n_gpu: 4, distributed training: False, 16-bits training: False
2019-04-13 09:32:43,361: loading vocabulary file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-vocab.txt from cache at /home/cloud/.pytorch_pretrained_bert/26bc1ad6c0ac742e9b52263248f6d0f00068293b33709fae12320c0e35ccfbbb.542ce4285a40d23a559526243235df47c5f75c197f04f37d1a0c124c32c9a084
2019-04-13 09:32:43,676: loading archive file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased.tar.gz from cache at /home/cloud/.pytorch_pretrained_bert/9c41111e2de84547a463fd39217199738d1e3deb72d4fec4399e6e241983c6f0.ae3cef932725ca7a30cdcb93fc6e09150a55e2a130ec7af63975a16c153ae2ba
2019-04-13 09:32:43,677: extracting archive file /home/cloud/.pytorch_pretrained_bert/9c41111e2de84547a463fd39217199738d1e3deb72d4fec4399e6e241983c6f0.ae3cef932725ca7a30cdcb93fc6e09150a55e2a130ec7af63975a16c153ae2ba to temp dir /tmp/tmprmsendyc
2019-04-13 09:32:48,700: Model config {
  "attention_probs_dropout_prob": 0.1,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "max_position_embeddings": 512,
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "type_vocab_size": 2,
  "vocab_size": 30522
}

2019-04-13 09:33:17,672: ***** Running training *****
2019-04-13 09:33:17,672:   Num examples = 0
2019-04-13 09:33:17,672:   Batch size = 32
2019-04-13 09:33:17,672:   Num steps = 0
2019-04-13 09:33:17,674: Loading training examples for epoch 0
Training examples: 0it [00:00, ?it/s]
Traceback (most recent call last):
  File "finetune_on_pregenerated.py", line 333, in <module>
    main()
  File "finetune_on_pregenerated.py", line 286, in main
    num_data_epochs=num_data_epochs)
  File "finetune_on_pregenerated.py", line 101, in __init__
    assert i == num_samples - 1  # Assert that the sample count metric was true
UnboundLocalError: local variable 'i' referenced before assignment
I found out the issue. The text corpus I was using is just one document. So the code is for two or more documents only?<|||||>Yes only for multiple documents.
We have a test now to check that since #478 thanks to @Rocketknight1.