Solved by tom in post #2 
              
              
                kl_div and the CPU backward are in aten/src/ATen/native/Loss.cpp, the cuda backward is in aten/src/ATen/native/cuda/Loss.cu. 
I once tried to write this up more generally: 
https://lernapparat.de/selective-excursion-into-pytorch-internals/ 
Best regards 
Thomas