yamatazen commited on
Commit
0c9418b
·
verified ·
1 Parent(s): dde7847

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +49 -48
README.md CHANGED
@@ -1,48 +1,49 @@
1
- ---
2
- base_model:
3
- - lemon07r/Gemma-2-Ataraxy-v4d-9B
4
- - UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3
5
- - ehristoforu/Gemma2-9B-it-psy10k-mental_health
6
- library_name: transformers
7
- tags:
8
- - mergekit
9
- - merge
10
-
11
- ---
12
- # Gemma2-Ataraxy-Psycho-9B
13
-
14
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
-
16
- ## Merge Details
17
- ### Merge Method
18
-
19
- This model was merged using the [DELLA](https://arxiv.org/abs/2406.11617) merge method using [UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3](https://huggingface.co/UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3) as a base.
20
-
21
- ### Models Merged
22
-
23
- The following models were included in the merge:
24
- * [lemon07r/Gemma-2-Ataraxy-v4d-9B](https://huggingface.co/lemon07r/Gemma-2-Ataraxy-v4d-9B)
25
- * [ehristoforu/Gemma2-9B-it-psy10k-mental_health](https://huggingface.co/ehristoforu/Gemma2-9B-it-psy10k-mental_health)
26
-
27
- ### Configuration
28
-
29
- The following YAML configuration was used to produce this model:
30
-
31
- ```yaml
32
- merge_method: della
33
- dtype: bfloat16
34
- out_dtype: bfloat16
35
- base_model: UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3
36
- models:
37
- - model: lemon07r/Gemma-2-Ataraxy-v4d-9B
38
- parameters:
39
- density: 0.6
40
- weight: 0.6
41
- - model: ehristoforu/Gemma2-9B-it-psy10k-mental_health
42
- parameters:
43
- density: 0.5
44
- weight: 0.3
45
- parameters:
46
- epsilon: 0.1
47
- lambda: 1.0
48
- ```
 
 
1
+ ---
2
+ base_model:
3
+ - lemon07r/Gemma-2-Ataraxy-v4d-9B
4
+ - UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3
5
+ - ehristoforu/Gemma2-9B-it-psy10k-mental_health
6
+ library_name: transformers
7
+ tags:
8
+ - mergekit
9
+ - merge
10
+
11
+ ---
12
+ ![image/png](https://huggingface.co/yamatazen/Gemma2-Ataraxy-Psycho-9B/resolve/main/Gemma2-Ataraxy-Psycho-9B.png?download=true)
13
+ # Gemma2-Ataraxy-Psycho-9B
14
+
15
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
16
+
17
+ ## Merge Details
18
+ ### Merge Method
19
+
20
+ This model was merged using the [DELLA](https://arxiv.org/abs/2406.11617) merge method using [UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3](https://huggingface.co/UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3) as a base.
21
+
22
+ ### Models Merged
23
+
24
+ The following models were included in the merge:
25
+ * [lemon07r/Gemma-2-Ataraxy-v4d-9B](https://huggingface.co/lemon07r/Gemma-2-Ataraxy-v4d-9B)
26
+ * [ehristoforu/Gemma2-9B-it-psy10k-mental_health](https://huggingface.co/ehristoforu/Gemma2-9B-it-psy10k-mental_health)
27
+
28
+ ### Configuration
29
+
30
+ The following YAML configuration was used to produce this model:
31
+
32
+ ```yaml
33
+ merge_method: della
34
+ dtype: bfloat16
35
+ out_dtype: bfloat16
36
+ base_model: UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3
37
+ models:
38
+ - model: lemon07r/Gemma-2-Ataraxy-v4d-9B
39
+ parameters:
40
+ density: 0.6
41
+ weight: 0.6
42
+ - model: ehristoforu/Gemma2-9B-it-psy10k-mental_health
43
+ parameters:
44
+ density: 0.5
45
+ weight: 0.3
46
+ parameters:
47
+ epsilon: 0.1
48
+ lambda: 1.0
49
+ ```