amankhandelia:

RuntimeError: Expected tensor to have CPU Backend, but got tensor with CUDA Backend (while checking arguments for cudnn_ctc_loss)


Unfortunately, the error doesn’t tell you which tensor should have a CPU backend.
cudnn wants the targets on the CPU (and I’d think the lengths, too).
I your second example probably uses the native implementation, but there, too, it is preferable to keep the lengths on the CPU (or else they’ll be moved to CPU and you have a sync point). You can tell which backend is used by printing loss.grad_fn and checking whether it includes “cudnn” in the grad_fn's name.
Best regards
Thomas