richard, I can’t seem to find many of the function names from the profiler in the pytorch code. For example, if I look at the five functions that take the most time:

_th_get_device                 : GPU time: 7100.00
LSTMFusedBackward              : GPU time: 7829.99
mm                             : GPU time: 7990.41
MmBackward                     : GPU time: 9270.85
add                            : GPU time: 9508.88

I find the string _th_get_device in aten/src/ATen/core/aten_interned_strings.h but there really isn’t a function definition there.
For LSTMFusedBackward, I don’t find that particular string by grepping, but I do find LSTMFused in torch/nn/_functions/thnn/auto.py - apparently LSTMFusedBackward is a function that’s generated in there? I’m guessing that a lot of these functions are actually generated in auto.py. MmBackward is the backwards mm, it seems (for example).