I have the same issue for profiling the runtime of each layer of a model, including forward, backward and update during training. Do you find some pytorch tools? Or give me some detailed implement hints. Thank you.