I’m using the CTCLoss in PyTorch version 1.0.0.
My vague understanding from the source and discussions I’ve read is that it wraps some external cpp modules (linking to cudnn), and implements its own backwards() rather than relying on pytorch’s autograd.
I therefore assume higher order gradients (e.g. HVPs) through CTCLoss won’t work.
For example, running:
# CTC Loss test
import torch

ctc_loss = torch.nn.CTCLoss()
log_probs = torch.randn(50, 16, 20).log_softmax(2).detach().requires_grad_()
log_probs.size()

targets = torch.randint(1, 20, (16, 30), dtype=torch.long)
targets.size()

input_lengths = torch.full((16,), 50, dtype=torch.long)
target_lengths = torch.randint(10,30,(16,), dtype=torch.long)

loss = ctc_loss(log_probs, targets, input_lengths, target_lengths)

(grads, ) = torch.autograd.grad(loss, log_probs, create_graph=True)  # ∇J
grads_summed = torch.sum(grads, (0,1,2))  # equilvant to ∇J⋅1
torch.autograd.grad(grads_summed, log_probs)  # attempt HVP ∇(∇J⋅1)

results in:

Runtime Error: derivative for _ctc_loss_backward is not implemented

Is there any way around this other than to take on the daunting task of implementing backward() for ctc_loss_backward?