Is it possible to train all the models available in hub using deepspeed?
If not, how to know which ones?
Specifically, I interested in sentence-transformers models. 
(is it possible at all to train them using transformers api?)

@nreimers @stas00 

thanks!




more or less yes, please see https://github.com/huggingface/transformers/pull/12695 - I don't have 100% coverage of all models but the majority of popular models work just fine as seen by the tests.

I'm not rushing to merge that PR since it's just many tests.

If you find any that don't work please report and we will make it work.<|||||>amazing. thanks for your help.