Support DeepSpeed for language modeling finetuning
# 🚀 Feature request

https://github.com/microsoft/DeepSpeed

This was just released, and given the code flow in `run_language_modeling.py` it seems like it would not be too difficult to drop-in, and it has a permissible license (MIT).

However, given the dependencies and difficulty installing them, it would likely have to be done in a separate file.

## Motivation

![](https://i.kym-cdn.com/entries/icons/mobile/000/006/360/gottago.jpg)
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
