The Windows task manager doesn’t show the “compute” tab by default, so either add it or use nvidia-smi.
Since the GPU Util is >0%, the GPU is used.
You might suffer from other bottlenecks, e.g. data loading, so make sure to store the data on a local SSD.
If that doesn’t help, you would have to profile the code and check, which parts uses most of the runtime.