Thank you for your answer, I think your answer about nvidia-smi is right.
But I have another question, could you please explain why for me?
We want to deploy more than one pytorch model on single GPU, So we care about peak usage more.I have monitored the usage of GPU by nvidia-smi or torch.cuda.memory_summary(), For resnet_maskrcnn nvdia-smi result is 3095MB, and torch.cuda.max_memory_reserved() result is 1778384896byte(1696MB). For mobilenetV3_maskrcnn nvidia-smi result is 5964MB, and torch.cuda.max_memory_reserved() result 4640997376byte(4426MB), both are bigger than resnet_maskrcnn, but we can deploy three mobilenetV3_maskrcnn on 1050ti, and only two resnet_maskrcnn on 1050ti. Could you explain why or there is somthing wrong in my profile method.
I have summary the result for a table: