No, they do not. I think the CPUs were mainly working on image I/O loading and pre-processing (including data augmentation). But I found even if CPUs were not fully occupying all of the 14 threads (some are not 100% though htop), it often happened that one GPU util (either) is ~90%, the other is ~20% and reversed in a few second. I will take a look at the visual profiler. Thanks for the recommendation.