I suspect that my dataloading is slowing things down because I’m reading from disk a lot and don’t have an SSD.
So I don’t actually care about which part of my pytorch model is slow, I just want that when I use cprofile or py-spy I get approximately accurate numbers regarding the ratio between how much time is spent running the model vs loading data.
But I remember reading about how by default when profiling CUDA code only the time to launch the kernel will be calculated, not the runtime on GPU. One has to synchronize to get more accurate results.
But I only care about how long running the entire model takes, and assume there has to be a synchronization step to get the final result?
TL;DR If I just care about comparing the runtime of forward-backward to nonpytorch code can I just use normal python profiling tools?