Not working on Spanish but I think I can share my experience here.

I suppose some of the samples are longer and I got unlucky in one batch.

I tried the trick here 4, it didn’t help. And I found out the main cause of OOM error is evaluating. The Trainer class saves all prediction outputs in GPU for faster evaluating. You can set eval_accumulation_steps to a small value(60 for me with batch size 8 on a 3090), but then it moves tensor from GPU to CPUs and it’s slow as hell. In the end I settled down with using 10% of testset when training and do the full evaluation at the end, still it adds 1-2 hours of overhead.

After I set up the model and invoke train , it takes a long time for training to actually start. I’m talking hours when using the complete common voice dataset (training and validation samples together).

Having the same issue here, in my understanding data collating is very cpu intense so I tried increase the num_worker and it’s not helping. I hope someone else has answers.

Iteration and hill-climbing are going to be very slow because of the reasons above.

Maybe you can disable evaluation completely, that should speed you up:-)
Default setting from the demo notebook should get you to somewhere decent. And the wer metric is slow as well. Someone should do a profiling.