Does anybody know how to implement label smoothing (LS) with CTCLoss? I found a lot of articles about CrossEntropyLoss with Label smoothing, but nothing about CTCLoss.
I’ve found a paper (https://arxiv.org/pdf/1711.02212.pdf 31) mentioning about the LS with CTCLoss, with an equivalent approach with KLDiv, but have no idea how to implement it with PyTorch. TIA!
PS:
I implemented the CTCLoss with Label smoothing like:
class SmoothCTCLoss(_Loss):

    def __init__(self, num_classes, blank=0, weight=0.01):
        super().__init__(reduction='mean')
        self.weight = weight
        self.num_classes = num_classes

        self.ctc = nn.CTCLoss(reduction='mean', blank=blank, zero_infinity=True)
        self.kldiv = nn.KLDivLoss(reduction='batchmean')

    def forward(self, log_probs, targets, input_lengths, target_lengths):
        ctc_loss = self.ctc(log_probs, targets, input_lengths, target_lengths)

        kl_inp = log_probs.transpose(0, 1)
        kl_tar = torch.full_like(kl_inp, 1. / self.num_classes)
        kldiv_loss = self.kldiv(kl_inp, kl_tar)

        #print(ctc_loss, kldiv_loss)
        loss = (1. - self.weight) * ctc_loss + self.weight * kldiv_loss
        return loss

Can anybody confirm if this is correct or not? I also referred to this:

  
      
      Lei Mao's Log Book
  
  
    

Label Smoothing 21

Hello Underworld.