Implement DeLighT: Very Deep and Light-weight Transformers
# 🌟 New model addition

## Model description
DeLight, that delivers similar or better performance than transformer-based models with significantly fewer parameters. DeLighT more efficiently allocates parameters both (1) within each Transformer block using DExTra, a deep and light-weight transformation and (2) across blocks using block-wise scaling, that allows for shallower and narrower DeLighT blocks near the input and wider and deeper DeLighT blocks near the output. Overall, DeLighT networks are 2.5 to 4 times deeper than standard transformer models and yet have fewer parameters and operation
https://arxiv.org/pdf/2008.00623.pdf

<!-- Important information -->

## Open source status

* [X] the model implementation is available: (give details)

It is available at https://github.com/sacmehta/delight

* [ ] the model weights are available: (give details)
* [X] who are the authors: (mention them, if possible by @gh-username)
@sacmehta
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
