Hi, I meet the same error.  I pad target labels with “PAD_ID=1”, and “BLANK_ID = 0” in my vocabulary.  After some batches, it only predicts blank.
def CTCLoss(ctc_decoder_out, samples):
    ctcprobs = ctc_decoder_out.log_softmax(-1)
    loss_func = torch.nn.CTCLoss(blank=0, reduction="mean", zero_infinity=False)
    ctc_target = samples["real_target"]
    src_lengths = samples["src_lengths"]
    tgt_lengths = ctc_target.ne(tgt_dict.pad()).sum(-1)
    ctc_loss = loss_func(ctcprobs, ctc_target.cpu(), src_lengths.cpu(), tgt_lengths.cpu())
    return ctc_loss

This is my code.  I don’t know weather it is hard to train ctc loss or there are something wrong in my code?  After one epoch, it stills only predict blank and the ctc loss is really slowly decreased. Could you help solve my problem by any chance? Thanks.