|
--- |
|
language: |
|
- en |
|
--- |
|
|
|
**Mistral Nemo 12B Lyra v2 - Stage 1** |
|
|
|
This is Stage 1 of Lyra v2. Only SFT and 2-Step Merge has been performed at this stage. |
|
|
|
This is still raw, there may be issues. |
|
|
|
Kinda wordy and verbose like Claude is, but that is fixable in stage 2, hopefully. |
|
|
|
Stage 2 Plans - RL currently TBD |
|
|
|
--- |
|
|
|
\- `[/INST]` may leak, let's say that is an unintended side-effect. Just simply add it to your custom stopping strings. I tried adding it as an EOS in model config but that ruined the model somehow, so I didn't. |
|
|
|
\- min_p value of 0.1 is a sane value to use. Temp? 0.7-1.2, up to you. |
|
|
|
\- Prompt Format: ChatML |
|
|
|
Have a day. |
