Update README.md
Browse files
README.md
CHANGED
|
@@ -1,34 +1,56 @@
|
|
| 1 |
-
|
| 2 |
-
base_model: []
|
| 3 |
-
library_name: transformers
|
| 4 |
-
tags:
|
| 5 |
-
- mergekit
|
| 6 |
-
- merge
|
| 7 |
|
| 8 |
-
|
| 9 |
-
# 24b
|
| 10 |
|
| 11 |
-
This is
|
| 12 |
|
| 13 |
-
|
| 14 |
-
### Merge Method
|
| 15 |
|
| 16 |
-
|
| 17 |
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
-
The following YAML configuration was used to produce this model:
|
| 26 |
|
| 27 |
-
|
| 28 |
-
base_model: /home/quixi/storage/models/Gryphe-Codex-24B-Small-3.2+/home/quixi/storage/models/NewEden-Austral-24B-Codex-Lora
|
| 29 |
-
dtype: bfloat16
|
| 30 |
-
merge_method: passthrough
|
| 31 |
-
models:
|
| 32 |
-
- model: /home/quixi/storage/models/Gryphe-Codex-24B-Small-3.2+/home/quixi/storage/models/NewEden-Austral-24B-Codex-Lora
|
| 33 |
|
| 34 |
-
|
|
|
|
| 1 |
+
# Wot is this
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
|
| 3 |
+
Just another checkpoint, better to use the -Winton model but this is released to keep in line with being an actual OSS Finetuner. (Unlike some others who don't release datasets or checkpoints!)
|
|
|
|
| 4 |
|
| 5 |
+
This is the SFT part of the MS3.2 train ontop of Codex
|
| 6 |
|
| 7 |
+
Wandb: https://wandb.ai/gum1h0x/austral/artifacts/axolotl-config/config-4hspge7d/v0/files/axolotl_config_ept225f_.yml
|
|
|
|
| 8 |
|
| 9 |
+
Datasets:
|
| 10 |
|
| 11 |
+
```
|
| 12 |
+
datasets:
|
| 13 |
+
- path: Delta-Vector/Hydrus-Claude-Instruct-2.7K
|
| 14 |
+
type: dan-chat-advanced
|
| 15 |
+
- path: Delta-Vector/Hydrus-Claude-Instruct-5K
|
| 16 |
+
type: dan-chat-advanced
|
| 17 |
+
- path: Delta-Vector/Orion-Shoujo-AI-Filtered-ShareGPT
|
| 18 |
+
type: dan-chat-advanced
|
| 19 |
+
- path: PocketDoc/Dans-Personamaxx-VN
|
| 20 |
+
type: dan-chat-advanced
|
| 21 |
+
- path: NewEden/LIMARP-Complexity
|
| 22 |
+
type: dan-chat-advanced
|
| 23 |
+
- path: NewEden/PIPPA-Mega-Filtered
|
| 24 |
+
type: dan-chat-advanced
|
| 25 |
+
- path: NewEden/OpenCAI-ShareGPT
|
| 26 |
+
type: dan-chat-advanced
|
| 27 |
+
- path: NewEden/Creative_Writing-Complexity
|
| 28 |
+
type: dan-chat-advanced
|
| 29 |
+
- path: NewEden/Light-Novels-Roleplay-Logs-Books-Oh-My-duplicate-turns-removed
|
| 30 |
+
type: dan-chat-advanced
|
| 31 |
+
- path: PocketDoc/Dans-Failuremaxx-Adventure-3
|
| 32 |
+
type: dan-chat-advanced
|
| 33 |
+
- path: NewEden/Books-V2-ShareGPT
|
| 34 |
+
type: dan-chat-advanced
|
| 35 |
+
- path: NewEden/Deepseek-V3-RP-Filtered
|
| 36 |
+
type: dan-chat-advanced
|
| 37 |
+
- path: NewEden/Final-Alpindale-LNs-ShareGPT
|
| 38 |
+
type: dan-chat-advanced
|
| 39 |
+
- path: NewEden/DeepseekRP-Filtered
|
| 40 |
+
type: dan-chat-advanced
|
| 41 |
+
- path: NewEden/RP-logs-V2-Experimental
|
| 42 |
+
type: dan-chat-advanced
|
| 43 |
+
- path: anthracite-org/kalo_opus_misc_240827
|
| 44 |
+
type: dan-chat-advanced
|
| 45 |
+
- path: anthracite-org/kalo_misc_part2
|
| 46 |
+
type: dan-chat-advanced
|
| 47 |
+
- path: NewEden/Storium-Prefixed-Clean
|
| 48 |
+
type: dan-chat-advanced
|
| 49 |
+
- path: Delta-Vector/Hydrus-AM-Thinking-IF
|
| 50 |
+
type: dan-chat-advanced
|
| 51 |
+
```
|
| 52 |
|
|
|
|
| 53 |
|
| 54 |
+
TYSM to Gum1hox for sponsering the run, Trained on 1xB200 for 30 hours
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
|
| 56 |
+
https://x.com/gum1h0x
|