Update README.md
Browse files
README.md
CHANGED
@@ -7,8 +7,8 @@ license: apache-2.0
|
|
7 |
This is an experimental version of LimaRP for [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) with
|
8 |
about 1800 training samples _up to_ 4k tokens length. Contrarily to the previously released "v3" version for Llama-2, this one does
|
9 |
not include a preliminary finetuning pass on several thousands short stories. Initial testing has shown Mistral to be capable of
|
10 |
-
generating on its own the kind of stories that were included there; its training data appears to be quite diverse and not
|
11 |
-
been filtered
|
12 |
|
13 |
Due to software limitations, finetuning didn't take advantage yet of the Sliding Window Attention (SWA) which would have allowed
|
14 |
to use longer conversations in the training data. Thus, this version of LimaRP should be considered an initial attempt and
|
|
|
7 |
This is an experimental version of LimaRP for [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) with
|
8 |
about 1800 training samples _up to_ 4k tokens length. Contrarily to the previously released "v3" version for Llama-2, this one does
|
9 |
not include a preliminary finetuning pass on several thousands short stories. Initial testing has shown Mistral to be capable of
|
10 |
+
generating on its own the kind of stories that were included there; its training data appears to be quite diverse and does not
|
11 |
+
seem to have been filtered for content type.
|
12 |
|
13 |
Due to software limitations, finetuning didn't take advantage yet of the Sliding Window Attention (SWA) which would have allowed
|
14 |
to use longer conversations in the training data. Thus, this version of LimaRP should be considered an initial attempt and
|