Wav2Vec2/XLRS-Wav2Vec2 Pre-Training
Dear 🤗-team,

I'd like to do pre-training with your implementation of Wav2Vec2 and/or XLRS-Wav2Vec2. I was wondering if there are any plans to add such scripts (or even a demo) to the repository?

PS: I already did pre-training in NVIDIA NeMo, but I'm having problems with porting my checkpoints. Being able to do everything within the Huggingface framework would be great.

Wav2Vec2 Pre-Training is more important.<|||||>I would also like to be able to pre-train a Wav2Vec2 model using my own raw audio files in a self-supervised way. It would be even better if I could use a pre-trained model as a starting point. Is there any way to do this currently?<|||||>@czonios Yes, you can fine-tune a Wav2Vec2 model! Please check [this blogpost](https://huggingface.co/blog/fine-tune-xlsr-wav2vec2) by @patrickvonplaten. 

Pre-training is not available as of now.<|||||>Hey, 

We should have Wav2Vec2 Pretraining added in ~2 weeks.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>Unstale<|||||>#11306 is under way I think<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.