If you are already pushing the input tensor as well as the model to the GPU via .cuda() or .to('cuda') then your GPU should already be used the the util. in nvidia-smi as well as the memory usage would also verify it. To further isolate your bottleneck, profile your code and try to narrow down which part of your code is the slowest.