Hello,
I had the same issue, I can give my solution but I don’t know if it will work for you.
So, in the pytorch documentation (CTCLoss — PyTorch 1.8.1 documentation 12), it is said that

The alignment of input to target is assumed to be “many-to-one”, which limits the length of the target sequence such that it must be ≤ the input length.

Sometimes the predicted segments’ length were smaller than the true ones, hence I had “inf” and “nan” during the training. To fix this, you need to allow zero_infinity :



zero_infinity ( bool  1 , optional ) – Whether to zero infinite losses and the associated gradients. Default: False Infinite losses mainly occur when the inputs are too short to be aligned to the targets.


You need to do that in your code :

model = Wav2Vec2ForCTC.from_pretrained(path_2_model)
model.config.ctc_zero_infinity = True

Hope it will solve your problem.
Best,
Omar