I see. Thank you. By the way, as I have run without blocking launches, I have got this big cudaSteamSychronize in green …


Is that just normally how Pytorch would work?


Does the runtime from the profiler representing the GPU and CPU runtime …?


I have actually run it with
nsys profile -t cuda,osrt,nvtx,cudnn,cublas -o baseline_1024_sup_photos_async -w true  -f true  python3 pythonfile.py 
Screenshot from 2021-02-05 18-31-501920×1080 266 KB