3ed0k4 commited on
Commit
390fe7e
·
verified ·
1 Parent(s): 3e6ec40

Upload folder using huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +51 -0
README.md ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - merge
5
+ - mergekit
6
+ - lazymergekit
7
+ - 3ed0k4/KRDModel_3ed0k4
8
+ - CorticalStack/pastiche-crown-clown-7b-dare-dpo
9
+ ---
10
+
11
+ # 3ed0k4_modelkrd
12
+
13
+ 3ed0k4_modelkrd is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
14
+ * [3ed0k4/KRDModel_3ed0k4](https://huggingface.co/3ed0k4/KRDModel_3ed0k4)
15
+ * [CorticalStack/pastiche-crown-clown-7b-dare-dpo](https://huggingface.co/CorticalStack/pastiche-crown-clown-7b-dare-dpo)
16
+
17
+ ## 🧩 Configuration
18
+
19
+ ```yaml
20
+ slices:
21
+ - sources:
22
+ - model: 3ed0k4/KRDModel_3ed0k4
23
+ layer_range:
24
+ - 0
25
+ - 32
26
+ - model: CorticalStack/pastiche-crown-clown-7b-dare-dpo
27
+ layer_range:
28
+ - 0
29
+ - 32
30
+ merge_method: slerp
31
+ base_model: 3ed0k4/KRDModel_3ed0k4
32
+ parameters:
33
+ t:
34
+ - filter: self_attn
35
+ value:
36
+ - 0
37
+ - 0.5
38
+ - 0.3
39
+ - 0.7
40
+ - 1
41
+ - filter: mlp
42
+ value:
43
+ - 1
44
+ - 0.5
45
+ - 0.7
46
+ - 0.3
47
+ - 0
48
+ - value: 0.5
49
+ dtype: bfloat16
50
+
51
+ ```