Always use SequentialSampler during evaluation
When evaluating, shouldn't we always use the SequentialSampler instead of DistributedSampler? Evaluation only runs on 1 GPU no matter what, so if you use the DistributedSampler with N GPUs, I think you'll only evaluate on 1/N of the evaluation set. That's at least what I'm finding when I run an older/modified version of this repo.
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2002?src=pr&el=h1) Report
> Merging [#2002](https://codecov.io/gh/huggingface/transformers/pull/2002?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/b0ee7c7df3d49a819c4d6cef977214bd91f5c075?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2002/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2002?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #2002   +/-   ##
=======================================
  Coverage   84.05%   84.05%           
=======================================
  Files         105      105           
  Lines       15555    15555           
=======================================
  Hits        13075    13075           
  Misses       2480     2480
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2002?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2002?src=pr&el=footer). Last update [b0ee7c7...508f939](https://codecov.io/gh/huggingface/transformers/pull/2002?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>Yes, this works! Thank you @ethanjperez.