paulilioaica commited on
Commit
2e80735
1 Parent(s): a8220b4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -37,13 +37,13 @@ by mainly adding the config in the `moe_mixtral.py` script from `mixtral` branch
37
 
38
  ```yaml
39
  base_model: rhysjones/phi-2-orange
40
- gate_mode: random
41
  dtype: float16
42
  experts:
43
  - source_model: cognitivecomputations/dolphin-2_6-phi-2
44
- positive_prompts: [""]
45
  - source_model: rhysjones/phi-2-orange
46
- positive_prompts: [""]
47
  ```
48
 
49
  ## 💻 Usage
 
37
 
38
  ```yaml
39
  base_model: rhysjones/phi-2-orange
40
+ gate_mode: cheap_embed
41
  dtype: float16
42
  experts:
43
  - source_model: cognitivecomputations/dolphin-2_6-phi-2
44
+ positive_prompts: ["research, logic, math, science"]
45
  - source_model: rhysjones/phi-2-orange
46
+ positive_prompts: ["programming, reasoning"]
47
  ```
48
 
49
  ## 💻 Usage