when the txt file has 5GB, a Killed prompt appears.
I am running run_language_modeling.py, 
python run_language_modeling.py \
	    --output_dir ${model_dir} \
	    --tokenizer_name $data_dir/wordpiece-custom.json \
	    --config_name $data_dir/config.json \
            --train_data_file "$data_dir/train.txt" \
            --eval_data_file $data_dir/valid.txt \
            --block_size=128 \
            --do_train \
            --per_device_train_batch_size 1 \
            --gradient_accumulation_steps 64 \
            --learning_rate 6e-4 \
            --weight_decay 0.01 \
            --adam_epsilon 1e-6 \
            --adam_beta1 0.9 \
            --adam_beta2 0.98 \
            --max_steps 500_000 \
            --warmup_steps 24_000 \
            --fp16 \
            --logging_dir ${model_dir}/tensorboard \
            --save_steps 1000 \
            --save_total_limit 20 \
            --seed 108 \
            --max_steps -1 \
            --num_train_epochs 20 \
            --overwrite_output_dir

when the txt file has 5GB, a Killed prompt appears.

What is your machine's specs? It's probably an out of memory error.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
