If you are profiling your script on the GPU, you could use nvtx ranges 2 and tag the multi head attention operation with it.
I don’t know, if the built-in profiler provides options to print layer-specific outputs, but you could also take a look at pyprof.