Suppose I have a model and it has only one layer, say the nn.linear layer. I am interested in finding out the acutal time of the operation i.e. matmul and bias adddition excluding the time to move the data around that is calls such as cudamalloc() or cudamallocmanaged().
What I plan of doing is finding out the cuda/cublas calls that nn.linear does and the instrument that call to get the time.
I have this idea but dont know how to execute this.
I have a NLP model(which I want to instrument) inside a container, which has pytorch built from nvidia ngc pytorch image. Any suggestions on how to do this?