I’m using libtorch (C++) and developing on Windows and I’m wanting to try and get more utilization out of my GTX970.
I gauge my GPU usage by using MSI Afterburner which also has a handy GPU memory usage graph.
My little project is a self-learning network ala Alpha Zero, so most of the time the network is spent generating the data rather than training on it. During these “generating” phases I’m mainly just calling “forward” and I notice that afterburner reports a usage of around 35-40%, but when I’m using “loss.backward()” and “optimizer.step()” the usage does spike up to 90%.
I tried to increase my usage by increase more threads that call forward in parallel but this didn’t increase my GPU usage. Nor did moving extra copies of the network onto the GPU and calling those separately help.
I’m not super familiar with the libtorch library just yet but is there a way to try to get more parallelism so that I can use more of my GPU during “generation?”
P.S. I once accidentally ran my .exe file twice and my GPU usage was 70% so is there perhaps a per process lock that’s limiting my GPU usage?