FastFormers into transformers
# 🌟 New model addition

## Model description

We just open-sourced [FastFormers](https://arxiv.org/abs/2010.13382) which are our SustaiNLP 2020 systems (FastFormers: Highly Efficient Transformer Models for Natural Language Understanding [paper](https://arxiv.org/abs/2010.13382)).

Currently, we are hosting this on our repository, but would like to merge it back to the transformers repository as an example.
our repo - https://github.com/microsoft/fastformers

For the purpose of the shared task, this is purely implemented with SuperGLUE data set.
So, it's dependent on Alex Wang(@W4ngatang)'s SuperGLUE data processing pipeline.
Also, many parts of the implementations are based on Alex'.
(https://github.com/W4ngatang/transformers/tree/superglue)

What would be the best way to merge this back?

## Open source status

* [x] the model implementation is available: https://github.com/microsoft/fastformers/blob/main/examples/fastformers/run_superglue.py
* [x] the model weights are available: demo systems are uploaded. https://github.com/microsoft/fastformers/releases/tag/v0.1-model
* [x] who are the authors: @ykim362 

Hi! This is great, thanks for offering to contribute it! From what I understand, `FastFormers` contains several scripts that can be applied to `transformers` models out of the box, that is, training, distillation, pruning, using quantization alongside the onnx runtime and fp16 optimizations. 

Is that correct? If that is so, the easiest way would be to add the corresponding scripts to the `examples/` directory, probably under `examples/fastformers`. If there are modifications made to the model themselves, we can take a look together at how we can integrate those in the library.<|||||>Hi, thanks for your interest! From what I understand, I think your model falls in the category of dynamic acceleration. For these types of paper, I recommend you to integrate it to `examples/`, just like [PABEE](https://github.com/huggingface/transformers/tree/master/examples/bert-loses-patience) and [DeeBERT](https://github.com/huggingface/transformers/tree/master/examples/deebert). I've emailed you an invitation to our Slack channel if it works for you. cc @LysandreJik <|||||> @LysandreJik yes, that is correct. Thanks @JetRunner, let's discuss more on the slack.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>I am sorry, but I have been fully loaded with some other stuffs. I won't be able to make a progress. I'd like to close this to avoid any confusion.