A low GPU utilization could be caused by various bottlenecks, e.g. such as the data loading.
You could profile different parts of the code to narrow it down further.
One approach would be to profile the entire strict via Nsight Systems as described here or via the PyTorch profiler.
Also, you could check the GPU util. by removing the DataLoader and by feeding CUDATensors directly to the model and you could profile the data loading standalone (i.e. without any model training) to see how long it takes to create each batch.