@ptrblck do you have any insight on why I am unable to achieve the expected 50% memory savings with AMP? Or a resource you could point me to where I can read up on why AMP doesn’t provide the expected memory savings in pytorch?