Test model outputs equivalence
Adds a test to check that the model outputs keep the same values and order as the tuple output.
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6445?src=pr&el=h1) Report
> Merging [#6445](https://codecov.io/gh/huggingface/transformers/pull/6445?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/96c3329f19f28e47eab7f9f20ed3504619e16722&el=desc) will **increase** coverage by `0.38%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6445/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6445?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6445      +/-   ##
==========================================
+ Coverage   79.95%   80.33%   +0.38%     
==========================================
  Files         153      153              
  Lines       27932    27928       -4     
==========================================
+ Hits        22332    22437     +105     
+ Misses       5600     5491     -109     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6445?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_longformer.py](https://codecov.io/gh/huggingface/transformers/pull/6445/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9sb25nZm9ybWVyLnB5) | `98.69% <100.00%> (+0.63%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/6445/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9iZXJ0LnB5) | `69.06% <0.00%> (-27.52%)` | :arrow_down: |
| [src/transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/6445/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iZXJ0LnB5) | `88.26% <0.00%> (-0.17%)` | :arrow_down: |
| [src/transformers/modeling\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/6445/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ194bG0ucHk=) | `91.23% <0.00%> (+0.21%)` | :arrow_up: |
| [src/transformers/modeling\_t5.py](https://codecov.io/gh/huggingface/transformers/pull/6445/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190NS5weQ==) | `83.71% <0.00%> (+0.37%)` | :arrow_up: |
| [src/transformers/modeling\_reformer.py](https://codecov.io/gh/huggingface/transformers/pull/6445/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19yZWZvcm1lci5weQ==) | `96.09% <0.00%> (+0.41%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/6445/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl90cmFuc2ZvX3hsLnB5) | `88.13% <0.00%> (+0.48%)` | :arrow_up: |
| [src/transformers/modeling\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/6445/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190cmFuc2ZvX3hsLnB5) | `79.69% <0.00%> (+0.56%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_flaubert.py](https://codecov.io/gh/huggingface/transformers/pull/6445/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9mbGF1YmVydC5weQ==) | `88.34% <0.00%> (+0.61%)` | :arrow_up: |
| ... and [14 more](https://codecov.io/gh/huggingface/transformers/pull/6445/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6445?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6445?src=pr&el=footer). Last update [96c3329...400c5ad](https://codecov.io/gh/huggingface/transformers/pull/6445?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>Awesome that we can remove the `cast_to_bool` hack here. Maybe we can remove it in `t5_modeling_tf_` as well<|||||>Side note, you should double-check the slow tests `test_saved_model_with_attentions_output` and `test_saved_model_with_hidden_states_output` still pass with the changes for the longformer model, as they are the ones that fail for t5 when we remove the `cast_to_bool`  thingy.<|||||>> Side note, you should double-check the slow tests `test_saved_model_with_attentions_output` and `test_saved_model_with_hidden_states_output` still pass with the changes for the longformer model, as they are the ones that fail for t5 when we remove the `cast_to_bool` thingy.

They did not pass with Longformer before as discussed with @jplu on the PR: https://github.com/huggingface/transformers/pull/5764#issuecomment-670002430, they should actually pass now I think :-) 