Hi, I am using profile to supervise the bottleneck of network.I find a weired phenomenon that, when I run a short code use cuda,I use ‘watch -n 1 nvidia-smi’ in another terminal,then I find that, four processes run on four cards one by one.
I find reason in origin codes, I find profiler_cuda.cpp, in which it will warmup all cuda devices in onEachDevice interface. And in record interface, it use cudaGetDevice, which returns the current device for the calling host thread.
And I am confused, if profiler use underlying thread on one single card, then why will run four processes on four cards?
And how profile deals with the four results?Use the host thread result?
And how to use profile in multi-cards? In which case recommend user to use multicards profile?
Are there more detailed information I can find or provide with me?
@ptrblck @albanD
Thank you very much!
May you a good day!