find cpu_amp is incorrect set, it's only set if self.sharded_ddp is n…
…ot None...


- trainer: @sgugger

amp does not work if self.sharded_ddp is None<|||||>_The documentation is not available anymore as the PR was closed or merged._<|||||>If Accelerate's support isn't handling mixed precision or if there is some issue with that, raise an issue with the reproducer. As of now, no changes are required to mixed precision handling.<|||||>> Now, mixed precision handling is moved to Accelerate. Only when Fairscale is being used mixed precision handling is supported in trainer as it isn't present in Accelerate.

Hi, I see in evaluation_loop and prediction. accelerate prepare is only called when  self.is_deepspeed_enabled and self.model_wrapped is self.model, so how about the non-deepspeed case?
         <|||||>> Hi, I see in evaluation_loop and prediction. accelerate prepare is only called when self.is_deepspeed_enabled and self.model_wrapped is self.model, so how about the non-deepspeed case?

That makes sense, thank you! I've raised a PR addressing this as linked above.<|||||>Hi, @pacman100 , I have another question. any idea of jit trace in evaluation in https://github.com/huggingface/transformers/blob/main/src/transformers/trainer.py#L1384, it also need the amp info.