I solved it. The reason GPUs are not fully utilized is that I have not used enough CPU.
The CPU has to pass the dataset to the GPU quickly, but I limited the number of CPUs in the DataLoader (num_workers=4), so the dataset was not fed to the GPU enough.
In fact, by increasing the num_workers(=16) of the DataLoader, the training speed was improved while fully utilizing the GPU.
Thank you.