Soumith Chintala had recently given a talk about PyTorch mentioning profiling.
He suggested to use your favorite Python profiler (e.g. Line_Profiler).
You can find the talk here 58.
Hope that helps you!