Nice, It could be interesting to visualize also:

  
      

      GitHub
  

  
    

GitHub - google-research/vmoe

  Contribute to google-research/vmoe development by creating an account on GitHub.