Solved by googlebot in post #2 
              
              
                Invoke backward() in autograd.profiler.profile context, instead of doing ad-hoc measurements. Then if you watch timeline from prof.export_chrome_trace(), you can maybe match operations with modules. I first thought that profiler.record_function() would allow tagging modules, but it seems to not work…