nvidia-smi will report the used device memory by all processes and will also show the allocated and reserved memory in PyTorch.
The memory saving depends on the actual model as well as the used algorithms and you could check the detailed memory stats via torch.cuda.memory_summary().