Why do we need to use `Loss.repeat(eval_batch_size)` in accelerator gather loop? 
https://github.com/huggingface/transformers/blob/b1198a8440cc05f569b0bc22038993a1e5e707ab/examples/pytorch/language-modeling/run_mlm_no_trainer.py#L510

If I do not use this, and simple do acclerator.gather(loss) my code is stuck at this point. 
But if I repeat the loss it seems to work. Can you explain why is this the case ?

Why do we also later use `losses = losses[: len(eval_dataset)]` ?

@sgugger Can you please help ?<|||||>Hi @thakursc1, Sylvain is currently off until next week - he'll answer your query when he's back from his break. Thanks for your understanding.<|||||>I'll look at why it does not work for a 0d tensor when I have a bit of bandwith (lots to do when coming back from vacation!)

The main reason we do it this way is to be able to compute the true average of the loss across all the processes at the end (otherwise we won't get the exact value of the loss). That being said, there is no reason why `accelerator.gather(loss)` in your code should be stuck. <|||||>https://github.com/huggingface/accelerate/pull/152 should fix the problem on the Accelerate side.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.