Update README.md
Browse files
README.md
CHANGED
@@ -7,10 +7,11 @@ license: apache-2.0
|
|
7 |
This is an experimental version of LimaRP for [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) with
|
8 |
about 1800 training samples _up to_ 4k tokens length. Contrarily to the previously released "v3" version for Llama-2, this one does
|
9 |
not include a preliminary finetuning pass on several thousands story. Initial testing has shown Mistral to be capable of
|
10 |
-
generating on its own the kind of stories that were included there
|
11 |
|
12 |
Due to software limitations, finetuning didn't take advantage yet of the Sliding Window Attention (SWA) which would have allowed
|
13 |
-
to use longer conversations in the training data. Thus, this version of LimaRP could be considered an initial attempt
|
|
|
14 |
|
15 |
For more details about LimaRP, see the model page for the [previously released version for Llama-2](https://huggingface.co/lemonilia/limarp-llama2-v2).
|
16 |
Most details written there apply for this version as well. Generally speaking, LimaRP is a longform-oriented, novel-style
|
|
|
7 |
This is an experimental version of LimaRP for [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) with
|
8 |
about 1800 training samples _up to_ 4k tokens length. Contrarily to the previously released "v3" version for Llama-2, this one does
|
9 |
not include a preliminary finetuning pass on several thousands story. Initial testing has shown Mistral to be capable of
|
10 |
+
generating on its own the kind of stories that were included there.
|
11 |
|
12 |
Due to software limitations, finetuning didn't take advantage yet of the Sliding Window Attention (SWA) which would have allowed
|
13 |
+
to use longer conversations in the training data. Thus, this version of LimaRP could be considered an initial attempt and
|
14 |
+
will be updated in the future.
|
15 |
|
16 |
For more details about LimaRP, see the model page for the [previously released version for Llama-2](https://huggingface.co/lemonilia/limarp-llama2-v2).
|
17 |
Most details written there apply for this version as well. Generally speaking, LimaRP is a longform-oriented, novel-style
|