Add support for HYBRID_SHARD and _HYBRID_SHARD_ZERO2 in the trainer
#21156 
_The documentation is not available anymore as the PR was closed or merged._<|||||>@sgugger   Requesting for review.<|||||>@pacman100 Gentle remainder.<|||||>Hello @raghavanone, after PR #23158, the FSDP logic is being handled by Accelerate. So, these changes should reflect in Accelerate if they aren't being supported by FSDP-XLA <|||||>> Hello @raghavanone, after PR #23158, the FSDP logic is being handled by Accelerate. So, these changes should reflect in Accelerate if they aren't being supported by FSDP-XLA

Sure, I can make the changes there too, any pointers on things to do in accelerate will of great help.<|||||>Hello @raghavanone, the above PR adds this functionality to the Accelerate which now powers Trainer—also made you the co-author of it. Thank you for all the effort and helping us keep up-to-date with the PyTorch FSDP.