Hello everyone, this is my first post on the Pytorch forums - I hope you are keeping well!
I understand that what I am going to ask is detailed and/or advanced. But please bare with me - I am willing to go all the way 
I want to track what (layer module) accessed my tensor and when my tensor was accessed.
For example - when training:

A tensor is used in the forward computation of a layer.
1a) The output produced by that layer is then saved for autograd on the later backwards pass.
Then during the backpass of the autograd, that same tensor is used to calculate gradients for the layer that produced it.

By recording when steps 1) and 2) are started, a tensor access pattern can be observed. The goal of this is to identify the time difference between the 1st and 2nd accesses. (I am a grad student - please feel free to ask more about my work)
Please understand that I have spent some time learning the Pytorch internals (i recommend here or here) and have thought of some methods of implementation.
I am posting here just to hear from some other voices who are more knowledgable and experienced in Pytorch.
Thank you for your time.