I was under the impression that Sean’s warpctc always assumes that you do loss.backward() directly on the CTC loss, i.e. the gradient_out of loss is 1, which is the same as not reducing and using loss.backward(torch.ones_like(loss)). Now if you do funny things loss2 = exp(-5*loss) + something_else, you will have different values backpropagating.
Best regards
Thomas