MemoryError: when we run_language_model.py to train an English Adapter
run run_language_model.py
https://github.com/Adapter-Hub/adapter-transformers/blob/master/examples/contrib/legacy/run_language_modeling.py
in such codes:

python3 run_language_modeling.py \
--output_dir=/mnt/localdata/cao/output_language_adapter_en/ \
--model_type=bert \
--model_name_or_path=bert-base-multilingual-cased \
--do_train \
--train_data_file=/mnt/localdata/cao/data_for_model/EN_train_updated.txt \
--do_eval \
--eval_data_file=/mnt/localdata/cao/data_for_model/EN_valid.txt \
--mlm \
--language en \
--train_adapter \
--adapter_config pfeiffer \
--per_gpu_train_batch_size 4 \
--per_gpu_eval_batch_size 4 \
--learning_rate 5e-5 \
--dataloader_num_workers 32 \
--cache_dir /mnt/localdata/cao/en_cache_dir/ 

get error:

with open(file_path, encoding="utf-8") as f:
text = f.read()
MemoryError

the train_data_file is around 6GB, I think it is the loading problem.
So how can we load large files?
You could try the another one  [run_language_modeling.py](https://github.com/Adapter-Hub/adapter-transformers/blob/master/examples/contrib/legacy/run_language_modeling.py) with `line-by-line` and add a `batch_size` along with `batched=True` to eliminate the memory error:

```
        tokenized_datasets = datasets.map(
            ...
            batched=True,
            batch_size=200,
            ...
        )
```<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.