Update README.md
Browse files
README.md
CHANGED
@@ -26,11 +26,11 @@ license: apache-2.0
|
|
26 |
- **16k Sequence Length** This model was trained with a 16k sequence length.
|
27 |
|
28 |
### Experimental Nature
|
29 |
-
Please note that Mistral-22b is still in a WIP.
|
30 |
|
31 |
### Upcoming Release: V.3
|
32 |
-
-
|
33 |
-
- I have done some preliminary results with my different base model, and it appears to achieve a lower loss after the first epoch compared to the other base model used for v0.1 and v0.2. so we have started training v0.3 with the new base model and with the longer dataset.
|
34 |
|
35 |
|
36 |
### Stay Updated
|
|
|
26 |
- **16k Sequence Length** This model was trained with a 16k sequence length.
|
27 |
|
28 |
### Experimental Nature
|
29 |
+
Please note that Mistral-22b is still in a WIP. v0.3 has started training now, with a different method than used before, this is to hopefully make the model more round in its internel knowlledge. Through my testing I found V2 to be a significant improvement over v.1.
|
30 |
|
31 |
### Upcoming Release: V.3
|
32 |
+
- v-0.3 will feature a different base model for testing purposed, however this model is pretty darn good for a second test. :)
|
33 |
+
- I have done some preliminary results with my different base model, and it appears to achieve a lower loss after the first epoch compared to the other base model used for v0.1 and v0.2. so we have started training v0.3 with the new base model and with the longer dataset, will be done and released in the next 48 hours.
|
34 |
|
35 |
|
36 |
### Stay Updated
|