For Roberta pretraining, how to enable large batch training using gradient accumulation?
In the example code, where can I enable gradient accumulation for large batch size training. Thanks. https://github.com/huggingface/transformers/tree/master/examples/language-modeling
Hi! You can use the `--gradient_accumulation_steps=N_STEPS` argument to the `run_language_modeling.py` script for that.

You can see all available flags by doing `python run_language_modeling.py --help`