Owen Arliawan
commited on
Commit
•
ac8a8ff
1
Parent(s):
6b6df25
Update README.md
Browse files
README.md
CHANGED
@@ -11,7 +11,7 @@ Trained on 2048 sequence length, while the base model is 8192 sequence length. F
|
|
11 |
|
12 |
Trained using Cognitive Computations Eric Hartford's https://huggingface.co/datasets/cognitivecomputations/dolphin dataset as we've found great results from their dolphin models in previous Llama models.
|
13 |
|
14 |
-
Trained for 2 days on 2x RTX3090 on our own machine, using 4-bit loading and Qlora 64-rank 128-alpha resulting in 2% trainable weights.
|
15 |
|
16 |
The goal for this model is to have the model less-censored and great at general tasks like the previous dolphin models by Eric Hartford.
|
17 |
We started training this BEFORE they launched their own full weight trained Llama-3-8B-Dolphin-2.9 with their own curated datasets and the newer "Dolphin 2.9" dataset.
|
@@ -31,6 +31,12 @@ Instruct format:
|
|
31 |
{{ user_message_2 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
|
32 |
```
|
33 |
|
|
|
|
|
|
|
|
|
|
|
|
|
34 |
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
35 |
|
36 |
Axolotl Config:
|
|
|
11 |
|
12 |
Trained using Cognitive Computations Eric Hartford's https://huggingface.co/datasets/cognitivecomputations/dolphin dataset as we've found great results from their dolphin models in previous Llama models.
|
13 |
|
14 |
+
Trained for 2 days on 2x RTX3090 on our own machine, using 4-bit loading and Qlora 64-rank 128-alpha resulting in ~2% trainable weights.
|
15 |
|
16 |
The goal for this model is to have the model less-censored and great at general tasks like the previous dolphin models by Eric Hartford.
|
17 |
We started training this BEFORE they launched their own full weight trained Llama-3-8B-Dolphin-2.9 with their own curated datasets and the newer "Dolphin 2.9" dataset.
|
|
|
31 |
{{ user_message_2 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
|
32 |
```
|
33 |
|
34 |
+
Quants:
|
35 |
+
|
36 |
+
GGUF:
|
37 |
+
|
38 |
+
ExllamaV2:
|
39 |
+
|
40 |
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
41 |
|
42 |
Axolotl Config:
|