It could be really nice if we could launch an experiment/initiative with the Model Garden, Federated teams and our community on training our first model garden model with our federated tools.
We have some recent interesting experiment with other frameworks.

  
      

      learning@home
  

  
    

Train vast neural networks together

  A library to train large neural networks across the internet. Imagine training one huge transformer on thousands of computers from universities, companies, and volunteers.


  

  
    
    
  

  


https://arxiv.org/abs/2106.10207

  

      huggingface.co
  

  
    

neuropark/sahajBERT · Hugging Face