Update context length
Browse files
README.md
CHANGED
@@ -7,7 +7,7 @@ pipeline_tag: text-generation
|
|
7 |
|
8 |
# OrcaMaid-13b-v2-FIX-32k
|
9 |
|
10 |
-
This is the fixed version of **OrcaMaid-v2-13b**,
|
11 |
|
12 |
This is a gradient SLERP merge of Microsoft's [Orca-2-13b](https://huggingface.co/microsoft/Orca-2-13b) and Undi and IkariDev's [Noromaid-v0.1.1-13b](https://huggingface.co/NeverSleep/Noromaid-13b-v0.1.1), biased towards Orca.
|
13 |
|
|
|
7 |
|
8 |
# OrcaMaid-13b-v2-FIX-32k
|
9 |
|
10 |
+
This is the fixed version of **OrcaMaid-v2-13b**, extended to `32768` context length via YaRN. The (now-deleted) v2 model had issues with the merged tokenizer that prevented it from stopping when necessary, and caused it to output broken ChatML tokens like `<|im_end`, etc.
|
11 |
|
12 |
This is a gradient SLERP merge of Microsoft's [Orca-2-13b](https://huggingface.co/microsoft/Orca-2-13b) and Undi and IkariDev's [Noromaid-v0.1.1-13b](https://huggingface.co/NeverSleep/Noromaid-13b-v0.1.1), biased towards Orca.
|
13 |
|