Delta-Vector commited on
Commit
a3577e2
·
verified ·
1 Parent(s): 09901ad

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +29 -25
README.md CHANGED
@@ -1,34 +1,38 @@
1
  ---
2
- base_model: []
3
  library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
  ---
9
- # 24b
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the Passthrough merge method using /home/quixi/storage/models/Austral-24B-Base + /home/quixi/storage/models/Austral-kto-lora as a base.
17
 
18
- ### Models Merged
19
 
20
- The following models were included in the merge:
21
 
22
-
23
- ### Configuration
24
-
25
- The following YAML configuration was used to produce this model:
26
 
27
  ```yaml
28
- base_model: /home/quixi/storage/models/Austral-24B-Base+/home/quixi/storage/models/Austral-kto-lora
29
- dtype: bfloat16
30
- merge_method: passthrough
31
- models:
32
- - model: /home/quixi/storage/models/Austral-24B-Base+/home/quixi/storage/models/Austral-kto-lora
33
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
34
  ```
 
1
  ---
2
+ base_model: Delta-Vector/Austral-24B-Base
3
  library_name: transformers
 
 
 
 
4
  ---
 
 
 
 
 
 
 
 
5
 
6
+ a KTO finetune ontop of the -Base Austral-24B, Still not recc'd for use, Use -Winton!
7
 
8
+ WandB: https://wandb.ai/new-eden/austral/artifacts/axolotl-config/config-v2nv3dlc/v0/files/axolotl_config_2u1b4uya.yml
9
 
10
+ Datasets:
 
 
 
11
 
12
  ```yaml
13
+ datasets:
14
+ - path: Delta-Vector/Tauri-IFeval-Dans-Tulu-KTO
15
+ split: train
16
+ type: chatml.argilla
17
+ - path: NewEden/Helpsteer-3-edit-kto-v7
18
+ split: train
19
+ type: chatml.argilla
20
+ - path: Delta-Vector/Tauri-Helpsteer-3-Preference-KTO
21
+ split: train
22
+ type: chatml.argilla
23
+ - path: NewEden/Helpsteer-3-edit-kto-v7
24
+ split: train
25
+ type: chatml.argilla
26
+ - path: Delta-Vector/Tauri-Opus-Accepted-GPT-Rejected-Opus-Writing-Prompts
27
+ split: train
28
+ type: chatml.argilla
29
+ - path: NewEden/Opus-accepted-hermes-rejected-shuffled
30
+ split: train
31
+ type: chatml.argilla
32
+ - path: NewEden/Purpura-Arkhaios-CC-KTO
33
+ split: train
34
+ type: chatml.argilla
35
+ - path: Delta-Vector/Tauri-KTO-Instruct-Mix
36
+ split: train
37
+ type: chatml.argilla
38
  ```