I just finished running the bottleneck on my script and I’m pretty lost with one section. There is a section called autograd profiler output (CUDA mode) and another called autograd profiler output (CPU mode). The top events are IndexBackward and index_put_impl. I’m not sure what those refer to but most importantly are the times that it lists for these.
In CPU mode it shows this:
Self CPU time total: 341.996s
CUDA time total: 0.000us
In CUDA mode it shows this:
Self CPU time total: 355.418s
CUDA time total: 15.872ms
What exactly are these times. Should I be concerned that in CUDA mode there is still so much time on the CPU.
This model runs very slowly with very high memory use and I’m only using a batch size of 16. Any more and it crashes.
Any help would be greatly appreciated.