Yes, autocast uses caching internally unless you manually disable it and the checks can be found here 2.



 vadimkantorov:

How do I ensure it does conversion of model parameters fp32->fp16 exactly once?


It’ll be done automatically for you if the parameter is found in the cache (which won’t be the case if e.g. you’ve modified it).