It would be interesting, the problem for us is that we cannot share data. However, we have been pretraining a large Wav2Vec2 at the KB for some time and right now, in the middle of training, it (marginally) outperforms the Swedish VoxPopuli. I’ll write something up and publish it before I go on vacation in two-three days. At least the license will be better than CC NonCommercial.

Screenshot 2021-06-15 at 16.01.481050×462 20 KB