Since the sprint is nearing the end, I uploaded my latest model checkpoint to the Hub: maxidl/wav2vec2-large-xlsr-german · Hugging Face 21 achieving a WER of 12.62%.
Based on the loss curves the training is not converged yet, so it might be worth going for even longer training runs. For now, I only managed to get 50k steps, taking around 30 hours on a single A100. Unfortunately, I did not have great success with distributed training.