Use nvidia-smi to see if the GPU is working at near 100%
if it is not, then there might be bottle neck somewhere, profile your code
If it is, your only option is to use smaller model so that it’s faster, or buy more GPUs, better GPUs