I also find the ctc backward is not consistent with the forward. It is kind of annoying that I find my network cannot converged, just because I do not use the log probability directly from the log_softmax function. I’d like to share a workaround if someone need the “true” gradient. CTCLoss gradient is incorrect · Issue #52241 · pytorch/pytorch · GitHub 1