thanks for your reply!
Does “small model” refer to all kind of model including LSTM, CNN and others?
I met the problem that sometimes 5-layer CNN can reach very high GPU utilization while a resnet18 network cannot. Is putting the Variable on GPU and using DataParallel() enough to  ensure the full utilization of GPUs?  or I just wondered whether there is a way to ensure the high utilization of GPUs?
Thanks again.