Gradient accumulation
Shouldn't we include `len(train_dataloader)` along with `step` here considering `len(train_dataloader)` might be a odd number? In that case, we could accumulate the gradients more times than `gradient_accumulation_steps`.
 
https://github.com/huggingface/transformers/blob/0412f3d9298cdb8ba7f69570753ec6a07d240c87/examples/run_squad.py#L231
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
