Sometimes one needs to manually use the gradient function, because the computed quantity is useful. For example, there is a paper 7 that applies reweighting to CTC loss via interpreting it as cross-entropy with some distribution (it happens that CTC’s gradient computes that distribution as an intermediate step).
Is there a way to compute/access the CTC loss gradient without resorting to backward hooks? (currently it’s hidden from manual calls if I understand well)
Can I do (efficiently) sth like:
# eq. 5 
def ctc_alignment_targets(logits, *args, dim = 1):
    log_probs = F.log_softmax(logits.detach(), dim = dim)
    ctc_grad, = torch.autograd.grad(F.ctc_loss(log_probs, *args), (logits,))
    return log_probs.exp() - ctc_grad

# eq. 13, but have not checked this equivalence in practice
# alignment_targets = ctc_alignment_targets(logits, ...)
# ctc_loss_via_cross_entropy = (alignment_targets * F.log_softmax(logits, dim = 1)).mean()

I can imagine some other ways to accomplish this (especially if modifications need not to be differentiated through), but is there a clean way to manually compute gradient of some known function?