Thank you again @tom, I understood the memory difference between my laptop and workstation.
Using memory_allocated() both my laptop and workstation allocate 438 MB GPU memory to tensors with AMP, and 435 MB without AMP.
But using memory_reserved() my laptop reserve 6.4 GB with AMP and 2.9 GB without AMP; the workstation reserves 1.63 GB with AMP and 2.79 GB without AMP.
So the AMP reduces Pytorch memory caching on Nvidia P100 (Pascal architecture) but increases memory caching on RTX 3070 mobile (Ampere architecture). I was expecting AMP to decrease memory allocation/reserved, not to increase it (or at least the same). As I saw in a thread that FP32 and FP16 tensors are not duplicated in GPU memory.
I also tried to train a ResNet-152 (224*224 images, batch size 32).  AMP is definitely slowing training on my laptop:
Laptop (minutes/epoch):
FP32: 2:02, TF32: 2:03, AMP: 2:20
Workstation (minutes/epoch):
FP32: 1:49, TF32: 1:46, AMP: 1:45
I was expecting TF32 and AMP to increase throughput on Ampere architecture, not to decrease it. Do you have any idea?