In nvidia-smi, Memory-Usage is how much GPU memory does this process use. GPU-Util reports what percentage of time one or more GPU kernel(s) was active for a given time perio. You say it seems that the training time isn’t different. Check GPU-Util.


In general, if you use BatchNorm, increasing the batchsize will lead to better results. Since the batchsize is increased by 9 times, if you still use the same hyperparameter configuration, such as num_epochs, you may need to check for overfitting (just a possible conjecture).