Thank you again ptrblck.
I only had a vage concept of data loading as just an easy task of feeding data to GPUs.
Thanks to your explanation, now I’m aware of that it involves lots of hardware aspects.
I also appreciate Ross Wightman’s writing, which you gave me a link to. The description is very informitive, and I could learn a lot about the bottleneck.
Now I suspect CPU power and storage I/O in my GCP settings. I will also try to learn how to imprement  a DistributedDataParallel in my environment for the program. Thank you.