Wav2Vec2 Conformer Encoder
# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

Implementing Conformer version of context encoder for [Wav2Vec2](https://github.com/pytorch/fairseq/blob/main/fairseq/models/wav2vec/wav2vec2.py#L390).



## Motivation

Fairseq has recently updated their Wav2Vec2 codebase, incorporating [Conformer-based](https://github.com/pytorch/fairseq/blob/main/examples/wav2vec/config/pretraining/wav2vec2_conformer_large_librivox.yaml) encoder as an option with already existing Transformer-based version. 

As long as a decent number of the new SOTA-matching papers are utilizing Conformers for either of ctc or transducer models, it would be a great addition to Transformers library to have an option of using transformer or conformer based encoder for [Wav2Vec2ForCTC](https://github.com/huggingface/transformers/blob/main/src/transformers/models/wav2vec2/modeling_wav2vec2.py#L1647)

Some of the papers: [JUST](https://arxiv.org/pdf/2111.08137v1.pdf), [Pushing the Limits of Semi-Supervised Learning for ASR](https://arxiv.org/pdf/2010.10504.pdf)



Thanks for letting me know - we should definitely add these conformer models asap. I can take care of this next Tuesday :-) <|||||>Also cc @anton-l <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.