Hi ! Thanks for the great tutorial. I fined-tuned a pre-trained model (facebook/wav2vec2-large-xlsr-53-french) and got encouraging results. However there are 2 things I wonder (and I think could maybe help some other fellow readers)


In both your notebook @patrickvonplaten and  @ozcangundes screenshots, and also from what I experienced, Validation Loss decreases during the first iterations but then starts going up while Wer keeps going down. Do you know why this is happening and can we be sure that WER is the best metric to track (for early stopping at least) especially as I use a custom LM for prediction once my model is done training?


Which are the main hyperparameters to change to try to improve performance (I have limit GPU time and any insights or educated guess on where to start could help ;))  ?


Thanks again!