torch.cuda.amp.autocast works in this way:

cast the layer into fp16 if the corresponding operation is fp16-safe.
https://pytorch.org/docs/stable/amp.html#autocast-op-reference 29
For example, batch normalization will stay in fp32.
run in the configured precision levels
If the layer input does not match the target precision, it will convert the type automatically.

Using mixed-precision could sometimes be slow due to the type-casting operation.
At inference time, using pure half-precision is faster than using amp.
But I’m not sure why your memory consumption is higher.
I always see less memory consumption with mixed-precision training on RTX 2080 Ti.
Maybe the above step 2 needs features in both fp32 and fp16 for precision switching layers.
You may want to refer to this paper 7.