Thanks for the reply. I have tried the profiler and graph visualization.
The profile of a network is too long, so I ran a simple net with single ‘neuron’ (but it’s still too long). So I uploaded an example computational graph.  Maybe it just because I used too many operations, and calling those ops takes so much time. I thought pytorch could do some optimizations like op fusions and reuse repeated subgraphs in the backward process, but maybe I didn’t use it right.

QQ截图20170929110910.jpg1252×939 448 KB