Awesome two people is enough - let’s create the project!
Think this is a really cool & interesting project and we should be able to demo it well! Here I think it’ll actually make most sense to fine-tune a pretrained Wav2Vec2 model, maybe this one: facebook/wav2vec2-base-100k-voxpopuli · Hugging Face 2 ?
There is currently no official fine-tuning script for Wav2Vec2, but it should be relatively easy to adapt the pretraining script: https://github.com/huggingface/transformers/pull/12271 4 or the official one in PyTorch.
Let’s create the project - very excited about this one