Thanks for the reply!
May I know if there is a RFC or plans for what kind of model parallelism will you support?
For example,  Model Parallelism — transformers 4.11.0.dev0 documentation 2 has described several kinds of model parallelism strategy, would you plan to support these kind of parallelism?