kl_div and the CPU backward are in aten/src/ATen/native/Loss.cpp 135, the cuda backward is in aten/src/ATen/native/cuda/Loss.cu.
I once tried to write this up more generally:
https://lernapparat.de/selective-excursion-into-pytorch-internals/ 61
Best regards
Thomas