Implement pytorch model with BetterTransformer in pytorch1.12?
### Feature request

BetterTransformer is a fastpath for the PyTorch Transformer API. The fastpath is a native, specialized implementation of key Transformer functions for CPU and GPU that applies to common Transformer use cases.

### Motivation

Faster.

### Your contribution

Pytorch official blog: https://pytorch.org/blog/a-better-transformer-for-fast-transformer-encoder-inference/
Hey @luoling1993, we're currently in talks with PyTorch to see how to best approach this. cc @erichan1<|||||>Hey there @luoling1993! I'm from the PyTorch team - we are working on this. My working PR https://github.com/erichan1/transformers/pull/2<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>Hey @luoling1993 !
This has been implemented now. Please check: https://medium.com/pytorch/bettertransformer-out-of-the-box-performance-for-huggingface-transformers-3fbe27d50ab2