[WIP] Add Theseus Compression
`transformers.theseus` provides the implementation for BERT-of-Theseus, LayerDrop and Mixout.

Original BERT-of-Theseus authors: @JetRunner @MichaelZhouwang

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5903?src=pr&el=h1) Report
> Merging [#5903](https://codecov.io/gh/huggingface/transformers/pull/5903?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/492bb6aa486856f8243dfeb533ed1b23e996e403?el=desc) will **decrease** coverage by `2.80%`.
> The diff coverage is `83.75%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5903/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5903?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5903      +/-   ##
==========================================
- Coverage   80.12%   77.31%   -2.81%     
==========================================
  Files         169      152      -17     
  Lines       32317    26290    -6027     
==========================================
- Hits        25893    20326    -5567     
+ Misses       6424     5964     -460     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5903?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/theseus/theseus\_list.py](https://codecov.io/gh/huggingface/transformers/pull/5903/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90aGVzZXVzL3RoZXNldXNfbGlzdC5weQ==) | `67.64% <67.64%> (ø)` | |
| [src/transformers/theseus/theseus\_module.py](https://codecov.io/gh/huggingface/transformers/pull/5903/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90aGVzZXVzL3RoZXNldXNfbW9kdWxlLnB5) | `88.88% <88.88%> (ø)` | |
| [src/transformers/theseus/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/5903/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90aGVzZXVzL19faW5pdF9fLnB5) | `100.00% <100.00%> (ø)` | |
| [src/transformers/theseus/layerdrop\_list.py](https://codecov.io/gh/huggingface/transformers/pull/5903/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90aGVzZXVzL2xheWVyZHJvcF9saXN0LnB5) | `100.00% <100.00%> (ø)` | |
| [src/transformers/theseus/mixout\_list.py](https://codecov.io/gh/huggingface/transformers/pull/5903/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90aGVzZXVzL21peG91dF9saXN0LnB5) | `100.00% <100.00%> (ø)` | |
| [src/transformers/theseus/theseus\_errors.py](https://codecov.io/gh/huggingface/transformers/pull/5903/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90aGVzZXVzL3RoZXNldXNfZXJyb3JzLnB5) | `100.00% <100.00%> (ø)` | |
| [src/transformers/commands/env.py](https://codecov.io/gh/huggingface/transformers/pull/5903/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb21tYW5kcy9lbnYucHk=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [src/transformers/commands/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/5903/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb21tYW5kcy9fX2luaXRfXy5weQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [src/transformers/commands/transformers\_cli.py](https://codecov.io/gh/huggingface/transformers/pull/5903/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb21tYW5kcy90cmFuc2Zvcm1lcnNfY2xpLnB5) | `0.00% <0.00%> (-87.50%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_mobilebert.py](https://codecov.io/gh/huggingface/transformers/pull/5903/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9tb2JpbGViZXJ0LnB5) | `23.38% <0.00%> (-73.68%)` | :arrow_down: |
| ... and [166 more](https://codecov.io/gh/huggingface/transformers/pull/5903/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5903?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5903?src=pr&el=footer). Last update [492bb6a...20dd34f](https://codecov.io/gh/huggingface/transformers/pull/5903?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>which examples have you tested this with?<|||||>> which examples have you tested this with?

run_glue and run_ner but more to come!<|||||>Want an early review for this? Would be wonderful! And it can save me some time before I do the docs. @sshleifer <|||||>> run_glue and run_ner but more to come! 

:heart_eyes: can't wait to re-fine-tune my NER models :hugs: <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Well I've been really busy recently but don't close it for me stalebot!<|||||>Thanks for reopening it @LysandreJik! I was stuck with some details and I'll probably get it done soon.<|||||>Sounds great, looking forward to it!<|||||>Will move to an independent package. Closing this.