When GPU training with automatic mixed precision, does forward pass use FP16, or FP32 accumulate? I’m asking this, because cheap GPUs have FP32 accumulate tensor core performance intentionally halved and NVidia claims that FP16 accumulate is okay for inference, but not for training, so I’m guessing, good enough for forward, but not backward pass. Would like to better assess  whether to get something like Titan RTX, or like 3090.