paulilioaica
commited on
Commit
•
2e80735
1
Parent(s):
a8220b4
Update README.md
Browse files
README.md
CHANGED
@@ -37,13 +37,13 @@ by mainly adding the config in the `moe_mixtral.py` script from `mixtral` branch
|
|
37 |
|
38 |
```yaml
|
39 |
base_model: rhysjones/phi-2-orange
|
40 |
-
gate_mode:
|
41 |
dtype: float16
|
42 |
experts:
|
43 |
- source_model: cognitivecomputations/dolphin-2_6-phi-2
|
44 |
-
positive_prompts: [""]
|
45 |
- source_model: rhysjones/phi-2-orange
|
46 |
-
positive_prompts: [""]
|
47 |
```
|
48 |
|
49 |
## 💻 Usage
|
|
|
37 |
|
38 |
```yaml
|
39 |
base_model: rhysjones/phi-2-orange
|
40 |
+
gate_mode: cheap_embed
|
41 |
dtype: float16
|
42 |
experts:
|
43 |
- source_model: cognitivecomputations/dolphin-2_6-phi-2
|
44 |
+
positive_prompts: ["research, logic, math, science"]
|
45 |
- source_model: rhysjones/phi-2-orange
|
46 |
+
positive_prompts: ["programming, reasoning"]
|
47 |
```
|
48 |
|
49 |
## 💻 Usage
|