[Reformer classification head] Implement the reformer model classification head for text classification
This PR is an effort to implement the classification head on top of the plain vanilla reformer model.
References have been taken from the Roberta and XLNet classification head to implement the reformer classification head.

Tried testing the implementation changes, but it is failing error, link to google collab https://colab.research.google.com/drive/1KFsQxLqsMB6vBF4_bRmTFGhdGwkgx0zI?usp=sharing;
Test scenario details;

- IMDB movie review dataset.

-  The cell3 has the reformer model classification head code.

- Since the length of the reviews is not more than 512 words( assuming there would be a few outliers, but that could be skipped)

- Initializing the model with the config.axial_pos_shape to (16,32) but the setting does not take effect and throws runtime error.

Please help.

Looks cool :-) Will take a look soon!<|||||># [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5198?src=pr&el=h1) Report
> Merging [#5198](https://codecov.io/gh/huggingface/transformers/pull/5198?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/0267668c3d648c6e41afda97f5df8671ee880ac3&el=desc) will **increase** coverage by `0.35%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5198/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5198?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5198      +/-   ##
==========================================
+ Coverage   77.01%   77.36%   +0.35%     
==========================================
  Files         128      146      +18     
  Lines       21615    25991    +4376     
==========================================
+ Hits        16646    20109    +3463     
- Misses       4969     5882     +913     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5198?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/5198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9fX2luaXRfXy5weQ==) | `99.24% <ø> (+0.11%)` | :arrow_up: |
| [src/transformers/benchmark/benchmark.py](https://codecov.io/gh/huggingface/transformers/pull/5198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9iZW5jaG1hcmsvYmVuY2htYXJrLnB5) | `74.01% <ø> (+5.16%)` | :arrow_up: |
| [src/transformers/benchmark/benchmark\_args.py](https://codecov.io/gh/huggingface/transformers/pull/5198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9iZW5jaG1hcmsvYmVuY2htYXJrX2FyZ3MucHk=) | `86.04% <ø> (+0.68%)` | :arrow_up: |
| [src/transformers/benchmark/benchmark\_args\_tf.py](https://codecov.io/gh/huggingface/transformers/pull/5198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9iZW5jaG1hcmsvYmVuY2htYXJrX2FyZ3NfdGYucHk=) | `87.50% <ø> (ø)` | |
| [src/transformers/benchmark/benchmark\_args\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9iZW5jaG1hcmsvYmVuY2htYXJrX2FyZ3NfdXRpbHMucHk=) | `89.13% <ø> (-7.75%)` | :arrow_down: |
| [src/transformers/benchmark/benchmark\_tf.py](https://codecov.io/gh/huggingface/transformers/pull/5198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9iZW5jaG1hcmsvYmVuY2htYXJrX3RmLnB5) | `61.53% <ø> (ø)` | |
| [src/transformers/benchmark/benchmark\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9iZW5jaG1hcmsvYmVuY2htYXJrX3V0aWxzLnB5) | `69.50% <ø> (-3.60%)` | :arrow_down: |
| [src/transformers/configuration\_albert.py](https://codecov.io/gh/huggingface/transformers/pull/5198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX2FsYmVydC5weQ==) | `100.00% <ø> (ø)` | |
| [src/transformers/configuration\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/5198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX2F1dG8ucHk=) | `93.18% <ø> (+0.32%)` | :arrow_up: |
| [src/transformers/configuration\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/5198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX2JhcnQucHk=) | `93.75% <ø> (+0.41%)` | :arrow_up: |
| ... and [113 more](https://codecov.io/gh/huggingface/transformers/pull/5198/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5198?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5198?src=pr&el=footer). Last update [223084e...d2f3839](https://codecov.io/gh/huggingface/transformers/pull/5198?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>I left a couple of comments that colud help you fix the PR :-) Also it would be great to add tests here<|||||>> I left a couple of comments that colud help you fix the PR :-) Also it would be great to add tests here

Could you please point to some reference code base, that I can look at. This will give me a good idea of the testing standard in the application.<|||||>Sure, here we go: https://github.com/huggingface/transformers/blob/1ae132a07d7f294cf58cd50f7db8723d00e282de/tests/test_modeling_bert.py#L361

and 

https://github.com/huggingface/transformers/blob/1ae132a07d7f294cf58cd50f7db8723d00e282de/tests/test_modeling_bert.py#L520

All this class should be added to `all_model_classes` in the Reformer test.<|||||>@patrickvonplaten I tried to fix the test case for the reformer classification head. But it is failing in one of the common test cases class,  
test_modelling_common.py-> test_hidden_states_output

where it tries to run the hidden states test for the Reformer Models, it fails only for classification head.
Since this is a common workflow, I am wondering where am I going wrong. Also, the newly added QA head on the reformer is not failing which is quite surprising. It looks like it bypasses the QA head and skips the test.

Please let me know your thoughts/suggestions.
Thanks
Amit<|||||>I think we are very close to merging this PR :-) After the minor changes as described above, I think we are good to merge<|||||>@patrickvonplaten Implemented the changes as suggested :)<|||||>Waiting for another approval - LGTM<|||||>Awesome PR @as-stevens, do you have a new example notebook?  If not I take it that it's possible to just import the class you created in your notebook and have added to the codebase.<|||||>@jstremme the link to the notebook, where I tried to fine-tune the model for IMDB text classification. Since there are not many pre-trained models for Reformer, I used the Crime and Punishment for classification. 
[https://colab.research.google.com/drive/1l61NccWTGMfNFPj1T8kvMjnik2iEQWfe](url)

The model did not perform well, as it is not a bi-directional trained model. The link [https://github.com/huggingface/transformers/issues/5023](url) contains more information.

Thanks
Amit<|||||>Thanks very much @as-stevens.  Great work, and I'm hopeful I'll be able to pretrain and use my own bi-directional Reformer for text classification by following a similar approach.  Happy to share high-level details with the transformers community as I go.

Unfortunately, the links you shared don't work for me.  Can you try updating?