run_summarization.py - num_update_steps_per_epoch calculation
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.11.0.dev0
- Platform: Linux-5.4.0-1055-azure-x86_64-with-glibc2.10
- Python version: 3.8.1
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): 2.5.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Disributed
### Who can help
 @Rocketknight1, @patrickvonplaten, @patil-suraj

**This is not a bug/issue specific to a model**  

Could some one help me understand why the number of update steps is 

        `num_update_steps_per_epoch = len(train_dataset) // training_args.per_device_train_batch_size`

and not:

        `num_update_steps_per_epoch = len(train_dataset) //  total_train_batch_size`

Since the batch_size is: total_train_batch_size =  training_args.per_device_train_batch_size * num_replicas


The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: run_summarization.py

## To reproduce


You are referring to this line here no? https://github.com/huggingface/transformers/blob/29dfb2dbb10cdba6327ff287db56b182c1db29b1/examples/tensorflow/summarization/run_summarization.py#L580 

So it's the tensorflow examples I assume. 

@Rocketknight1 - could you take a look here. I agree with @bpraveenk that  `num_update_steps_per_epoch = len(train_dataset) //  total_train_batch_size` would make more sense<|||||>@bpraveenk You are correct that this is a bug. I'll submit a PR to fix it soon, but in general this example is a little out-of-date and due to be partially rewritten. In the mean time if you want a more up-to-date TensorFlow summarization example, take a look at [this notebook](https://github.com/huggingface/notebooks/blob/master/examples/summarization-tf.ipynb).<|||||>Thank you @Rocketknight1 