How are you measuring the GPU utilization?
Could you use nvidia-smi for it?
I’m not using Windows, but have seen some posts here that show some options in your task manager (in case you use it), where you can select the “compute” option, while another option seem to be active by default.