Sao10K commited on
Commit
e896400
1 Parent(s): 3033a56

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -3
README.md CHANGED
@@ -5,14 +5,18 @@ language:
5
 
6
  **Mistral Nemo 12B Lyra v2 - Stage 1**
7
 
8
- This is Stage 1 of Lyra v2. Only SFT and Merge has been performed at this stage.
9
 
10
  This is still raw, there may be issues.
11
 
12
  Kinda wordy and verbose like Claude is, but that is fixable in stage 2, hopefully.
13
 
 
 
14
  ---
15
 
16
- \- `[/INST]` may leak, let's say that is unintended. Just simply add it to your custom stopping strings.
 
 
17
 
18
- \- min_p value of 0.1 is a sane value to use.
 
5
 
6
  **Mistral Nemo 12B Lyra v2 - Stage 1**
7
 
8
+ This is Stage 1 of Lyra v2. Only SFT and 2-Step Merge has been performed at this stage.
9
 
10
  This is still raw, there may be issues.
11
 
12
  Kinda wordy and verbose like Claude is, but that is fixable in stage 2, hopefully.
13
 
14
+ Stage 2 Plans - RL currently TBD
15
+
16
  ---
17
 
18
+ \- `[/INST]` may leak, let's say that is an unintended side-effect. Just simply add it to your custom stopping strings. I tried adding it as an EOS in model config but that ruined the model somehow, so I didn't.
19
+
20
+ \- min_p value of 0.1 is a sane value to use. Temp? 0.7-1.2, up to you.
21
 
22
+ \- Prompt Format: ChatML