Create README.md

README.md

@@ -0,0 +1,6 @@
+---
+language: en
+---
+This is a Hugging Face transformers-style conversion of the original __SMoE 15B-parameter__ model with __BFLOAT16__ from the paper "[Efficient Large Scale Language Modeling with Mixtures of Experts](https://arxiv.org/abs/2112.10684)" from Artetxe et al. The original model card can be found at https://github.com/facebookresearch/fairseq/blob/main/examples/moe_lm/model_card.md.
+
+The usage example and modeling code can be found at https://github.com/pingzhili/light-fairseq