In PyTorch CTC gradient takes log-probabilities as input (after log_softmax operation).
It seems that the CTC gradient formula in PyTorch (cpu version for simplicity 2) refers to and seems to be implementing eq. 16 from the original CTC paper 4.
Eq. 16 seems to compute gradients wrt logits, not log_probs, and PyTorch must compute gradients wrt log_probs, so it’s strange if they are using the same equation.
This mismatch confuses me, I must be missing something obvious because gradcheck passes well, so the formula in PyTorch must be correct…