Model parallelism + Adapters
Adding model parallelism for large T5 models and other models if needed.
Adding adapters (a generalization of #1289) at the same time.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
