Hugging Face's logo

k4yt3x
/
Cornerstone-0.1-LLaMA-70B

Text Generation
Transformers
Safetensors
llama
mergekit
Merge
conversational
text-generation-inference
Model card Files Community
k4yt3x commited on
Commit
7907c3a
·
verified ·
1 Parent(s): 7f9a4e0

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +1 -0
  2. README.md +45 -0
  3. config.json +40 -0
  4. mergekit_config.yml +8 -0
  5. model-00001-of-00062.safetensors +3 -0
  6. model-00002-of-00062.safetensors +3 -0
  7. model-00003-of-00062.safetensors +3 -0
  8. model-00004-of-00062.safetensors +3 -0
  9. model-00005-of-00062.safetensors +3 -0
  10. model-00006-of-00062.safetensors +3 -0
  11. model-00007-of-00062.safetensors +3 -0
  12. model-00008-of-00062.safetensors +3 -0
  13. model-00009-of-00062.safetensors +3 -0
  14. model-00010-of-00062.safetensors +3 -0
  15. model-00011-of-00062.safetensors +3 -0
  16. model-00012-of-00062.safetensors +3 -0
  17. model-00013-of-00062.safetensors +3 -0
  18. model-00014-of-00062.safetensors +3 -0
  19. model-00015-of-00062.safetensors +3 -0
  20. model-00016-of-00062.safetensors +3 -0
  21. model-00017-of-00062.safetensors +3 -0
  22. model-00018-of-00062.safetensors +3 -0
  23. model-00019-of-00062.safetensors +3 -0
  24. model-00020-of-00062.safetensors +3 -0
  25. model-00021-of-00062.safetensors +3 -0
  26. model-00022-of-00062.safetensors +3 -0
  27. model-00023-of-00062.safetensors +3 -0
  28. model-00024-of-00062.safetensors +3 -0
  29. model-00025-of-00062.safetensors +3 -0
  30. model-00026-of-00062.safetensors +3 -0
  31. model-00027-of-00062.safetensors +3 -0
  32. model-00028-of-00062.safetensors +3 -0
  33. model-00029-of-00062.safetensors +3 -0
  34. model-00030-of-00062.safetensors +3 -0
  35. model-00031-of-00062.safetensors +3 -0
  36. model-00032-of-00062.safetensors +3 -0
  37. model-00033-of-00062.safetensors +3 -0
  38. model-00034-of-00062.safetensors +3 -0
  39. model-00035-of-00062.safetensors +3 -0
  40. model-00036-of-00062.safetensors +3 -0
  41. model-00037-of-00062.safetensors +3 -0
  42. model-00038-of-00062.safetensors +3 -0
  43. model-00039-of-00062.safetensors +3 -0
  44. model-00040-of-00062.safetensors +3 -0
  45. model-00041-of-00062.safetensors +3 -0
  46. model-00042-of-00062.safetensors +3 -0
  47. model-00043-of-00062.safetensors +3 -0
  48. model-00044-of-00062.safetensors +3 -0
  49. model-00045-of-00062.safetensors +3 -0
  50. model-00046-of-00062.safetensors +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - NaniDAO/Llama-3.3-70B-Instruct-ablated
4
+ - huihui-ai/Llama-3.3-70B-Instruct-abliterated
5
+ - nbeerbower/Llama-3.1-Nemotron-lorablated-70B
6
+ - SentientAGI/Dobby-Unhinged-Llama-3.3-70B
7
+ - SicariusSicariiStuff/Negative_LLAMA_70B
8
+ library_name: transformers
9
+ tags:
10
+ - mergekit
11
+ - merge
12
+
13
+ ---
14
+ # Cornerstone-0.1-LLaMA-70B
15
+
16
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
17
+
18
+ ## Merge Details
19
+ ### Merge Method
20
+
21
+ This model was merged using the [SCE](https://arxiv.org/abs/2408.07990) merge method using [huihui-ai/Llama-3.3-70B-Instruct-abliterated](https://huggingface.co/huihui-ai/Llama-3.3-70B-Instruct-abliterated) as a base.
22
+
23
+ ### Models Merged
24
+
25
+ The following models were included in the merge:
26
+ * [NaniDAO/Llama-3.3-70B-Instruct-ablated](https://huggingface.co/NaniDAO/Llama-3.3-70B-Instruct-ablated)
27
+ * [nbeerbower/Llama-3.1-Nemotron-lorablated-70B](https://huggingface.co/nbeerbower/Llama-3.1-Nemotron-lorablated-70B)
28
+ * [SentientAGI/Dobby-Unhinged-Llama-3.3-70B](https://huggingface.co/SentientAGI/Dobby-Unhinged-Llama-3.3-70B)
29
+ * [SicariusSicariiStuff/Negative_LLAMA_70B](https://huggingface.co/SicariusSicariiStuff/Negative_LLAMA_70B)
30
+
31
+ ### Configuration
32
+
33
+ The following YAML configuration was used to produce this model:
34
+
35
+ ```yaml
36
+ base_model: huihui-ai/Llama-3.3-70B-Instruct-abliterated
37
+ merge_method: sce
38
+ dtype: float32
39
+ models:
40
+ - model: SicariusSicariiStuff/Negative_LLAMA_70B
41
+ - model: NaniDAO/Llama-3.3-70B-Instruct-ablated
42
+ - model: SentientAGI/Dobby-Unhinged-Llama-3.3-70B
43
+ - model: nbeerbower/Llama-3.1-Nemotron-lorablated-70B
44
+
45
+ ```
config.json ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "huihui-ai/Llama-3.3-70B-Instruct-abliterated",
3
+ "architectures": [
4
+ "LlamaForCausalLM"
5
+ ],
6
+ "attention_bias": false,
7
+ "attention_dropout": 0.0,
8
+ "bos_token_id": 128000,
9
+ "eos_token_id": [
10
+ 128001,
11
+ 128008,
12
+ 128009
13
+ ],
14
+ "head_dim": 128,
15
+ "hidden_act": "silu",
16
+ "hidden_size": 8192,
17
+ "initializer_range": 0.02,
18
+ "intermediate_size": 28672,
19
+ "max_position_embeddings": 131072,
20
+ "mlp_bias": false,
21
+ "model_type": "llama",
22
+ "num_attention_heads": 64,
23
+ "num_hidden_layers": 80,
24
+ "num_key_value_heads": 8,
25
+ "pretraining_tp": 1,
26
+ "rms_norm_eps": 1e-05,
27
+ "rope_scaling": {
28
+ "factor": 8.0,
29
+ "high_freq_factor": 4.0,
30
+ "low_freq_factor": 1.0,
31
+ "original_max_position_embeddings": 8192,
32
+ "rope_type": "llama3"
33
+ },
34
+ "rope_theta": 500000.0,
35
+ "tie_word_embeddings": false,
36
+ "torch_dtype": "float32",
37
+ "transformers_version": "4.49.0",
38
+ "use_cache": true,
39
+ "vocab_size": 128256
40
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ base_model: huihui-ai/Llama-3.3-70B-Instruct-abliterated
2
+ merge_method: sce
3
+ dtype: float32
4
+ models:
5
+ - model: SicariusSicariiStuff/Negative_LLAMA_70B
6
+ - model: NaniDAO/Llama-3.3-70B-Instruct-ablated
7
+ - model: SentientAGI/Dobby-Unhinged-Llama-3.3-70B
8
+ - model: nbeerbower/Llama-3.1-Nemotron-lorablated-70B
model-00001-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2073cef28afd5057812d2c068f5c0a9463f18709f41198dff8eef2e1893b0a25
3
+ size 4202692736
model-00002-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7ca74ebe02d6021361bae82b3aaec01aadac9634695b6a9157caec67136723e1
3
+ size 4202725624
model-00003-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a024b07834c73d22c8f776518e915e811c7e46aa66b95984c498599e1943b571
3
+ size 4362142872
model-00004-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc265c8729102049a5049cb008971aac233c99f7b679836c47f986dec75f94ee
3
+ size 4362142880
model-00005-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:da8e91e535b87ec78da30e428e19edcb39e9cc8b68ac9ecca12a079ee9be17ae
3
+ size 4966188888
model-00006-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:06081560e3e1f059b1e6aee84d5b05dc34d45289f4496caceba68739fe11b41a
3
+ size 4362142888
model-00007-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77bc1e3fa22c17c65c8ea69fc03e70818128f33aaabb9c06ff106d063420d241
3
+ size 4362142888
model-00008-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2a1e702fbccc209944ac4383fec39e6c557f16b41a29d96784299119890211f4
3
+ size 4966188888
model-00009-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a7f181597462fdd54e755ee0c29fa3e62b68a41c4dfb97ee6bf318f63218566f
3
+ size 4362142888
model-00010-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6b551e41e21267deb665beb7a597ee568fc4384bc731f7cd636002eda078c988
3
+ size 4362142888
model-00011-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:90119e5c0f2595b118ca834242a4945b92a672234f111b8b61bee0d4c48b0aa9
3
+ size 4966188888
model-00012-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a369efcfbf6d56e549ec34714aa2b08fbb0bc1e8d1dbce2a331eb62dfc82bf9
3
+ size 4362142880
model-00013-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e56195bb2adadf6a5cbaca761ba4116f28319a206ed9db0a94541809d2627179
3
+ size 4362142888
model-00014-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d1c7c71a958412ee95bbd3489da4ff21eb85e699a9a1e84ce6b6005a3d88e01
3
+ size 4966188888
model-00015-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:108098063eee848b12e120a84f59919ab85b2605a2c80bbddbff8aab686ff90c
3
+ size 4362142888
model-00016-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af5b2df5951d8145f8530715a2d9749f0a6c0a1cf5d6b32cf465609d570a6293
3
+ size 4362142888
model-00017-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:00cb1b87c7a36dab58198594234d1a86427e8d2296d15b0ab28bbd3afd48eaae
3
+ size 4966188888
model-00018-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:501902ecce966c1133d205edefa0d94a355d05a229284b83868a8a700513216e
3
+ size 4362142888
model-00019-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:38bd876c2afba75b1115bfe9fed0d0ef161c81c91c3ff10e7e6fb98589799e44
3
+ size 4362142888
model-00020-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1c2daa7e1dee3a43893f33b638667467a1f03f033df741566dcc88d4b33ba1f5
3
+ size 4966188880
model-00021-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9dac80f67ec868608bbf232380e6fe62638b9fbcf5ada5fe08b7c5f83fdbfb1b
3
+ size 4362142888
model-00022-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a2f3b583fde984b6e758f75cee78cb1abef33e69b968bf333c320fc41339c54f
3
+ size 4362142888
model-00023-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:abe9ec4011af65459d6a745e71638ab8e14615a06713053d2ed4c9a2d93ea2b2
3
+ size 4966188888
model-00024-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7aab762bcbc306eaf5fd1e2194dd1c3f0e9b4ad3a1ac50da13354d91e9b888b6
3
+ size 4362142888
model-00025-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a3636a1e4e9e382655587d7332ee776510cc13bf05c6812200afe2cc3ddde881
3
+ size 4362142888
model-00026-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bf0b09783bd001d38e5c3bead2d06f686db30120ae16bed744a00113f2d245f7
3
+ size 4966188888
model-00027-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f9d69440aaaac26ed4f6c8c6c9198222ad79787db25402d70ca8f97416ab8f2e
3
+ size 4362142888
model-00028-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4db834c0a56ac0891f8b35f30b0a81644a101727e851efa88bc953a26b64f064
3
+ size 4362142880
model-00029-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e6144576ef6e29fea7a7b79e94da3a8c8fa4ae5fe88c3a7df1fa60d9534d63b6
3
+ size 4966188888
model-00030-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0735bf8ebdec7128e09e969bc347e508e9458f281dbbcd60a9ac061d223ce528
3
+ size 4362142888
model-00031-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:034881e3720b0eb79e506f239ba1273c96a45f1026cc6d055e004d142c2eebd5
3
+ size 4362142888
model-00032-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:61d506c49fb05f1d8a46341d5ef25da20fc16c07fda0c5eab4e7bdcdd29b17f1
3
+ size 4966188888
model-00033-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c392dda46768b8cbc97b37099f5230ac89f3b2894f7a3de303fdc2c5c497cf76
3
+ size 4362142888
model-00034-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b40c18e810e7d450d0fe001ef7d968587b4d6c884cb1248db0e750bd5ba06f7f
3
+ size 4362142888
model-00035-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cafcf16de1968a82051c2883feb29111799f204fef6032e258347402bbfaf287
3
+ size 4966188888
model-00036-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:08041dbf9c10182e5b6f5c8a7b2d0493488f9981587840989f117c94f3d8f6ba
3
+ size 4362142880
model-00037-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:143734ff56745ced89b0ca9f60da662e7c484e1af5d62be753fb90514f3819d2
3
+ size 4362142880
model-00038-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:85084fcea04a4883d3a83f296f5fcc010b247cfe0c11329dcae6a71949abc525
3
+ size 4966188888
model-00039-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:80a1b98d21ea213c0ee55a7972ce8964bf50c702340b4b9924278c9ca5f7802a
3
+ size 4362142888
model-00040-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bee66d06c5f22b84a71e56fedb160d4f08bc9b1fcccf577946941d8a7f28187f
3
+ size 4362142888
model-00041-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9a41129ff14b573ec3a77c1688d3ce8dc536d4febc66f69a1ad79ac9303399c0
3
+ size 4966188888
model-00042-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0b0f6375b24c952a85f2d76cd4a11a9ab42afb8810b0e9080b8be15334e98d41
3
+ size 4362142888
model-00043-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:50c292515cc453b2795255964f2cafac0c69c65d5825568b9492202311317b86
3
+ size 4362142888
model-00044-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:07e6cf995d3d78a8c13077973e2a939df327e0cd09bf3275abbd47c9f00c116a
3
+ size 4966188888
model-00045-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8bf3d5fc78269aef44724b8e680a2ae6e2ab67dc4294c09c9ea9b0314725d605
3
+ size 4362142880
model-00046-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2aae2bf7069a8f8c7cef530fd55098343d84f7586b25eb45569a8ed3be25f875
3
+ size 4362142888