paulilioaica
/

PhiMiX-2x2B

Text Generation

Mixture of Experts

cognitivecomputations/dolphin-2_6-phi-2

rhysjones/phi-2-orange

Inference Endpoints

Model card Files Files and versions Community

paulilioaica commited on Jan 29

Commit

a8220b4

•

1 Parent(s): 400fb55

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -23,7 +23,7 @@
   "n_inner": null,
   "n_layer": 32,
   "n_positions": 2048,
-  "num_experts_per_tok": 2,
   "num_local_experts": 2,
   "resid_pdrop": 0.1,
   "rotary_dim": 32,

   "n_inner": null,
   "n_layer": 32,
   "n_positions": 2048,
+  "num_experts_per_tok": 1,
   "num_local_experts": 2,
   "resid_pdrop": 0.1,
   "rotary_dim": 32,