Hugging Face's logo

MidoriUnko
/
Behemoth-v2.2-Magnum-v4-123B

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
MidoriUnko commited on
Commit
85eb1bb
·
verified ·
1 Parent(s): 8961051

Upload 59 files

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +40 -0
  2. config.json +27 -0
  3. mergekit_config.yml +8 -0
  4. model-00001-of-00051.safetensors +3 -0
  5. model-00002-of-00051.safetensors +3 -0
  6. model-00003-of-00051.safetensors +3 -0
  7. model-00004-of-00051.safetensors +3 -0
  8. model-00005-of-00051.safetensors +3 -0
  9. model-00006-of-00051.safetensors +3 -0
  10. model-00007-of-00051.safetensors +3 -0
  11. model-00008-of-00051.safetensors +3 -0
  12. model-00009-of-00051.safetensors +3 -0
  13. model-00010-of-00051.safetensors +3 -0
  14. model-00011-of-00051.safetensors +3 -0
  15. model-00012-of-00051.safetensors +3 -0
  16. model-00013-of-00051.safetensors +3 -0
  17. model-00014-of-00051.safetensors +3 -0
  18. model-00015-of-00051.safetensors +3 -0
  19. model-00016-of-00051.safetensors +3 -0
  20. model-00017-of-00051.safetensors +3 -0
  21. model-00018-of-00051.safetensors +3 -0
  22. model-00019-of-00051.safetensors +3 -0
  23. model-00020-of-00051.safetensors +3 -0
  24. model-00021-of-00051.safetensors +3 -0
  25. model-00022-of-00051.safetensors +3 -0
  26. model-00023-of-00051.safetensors +3 -0
  27. model-00024-of-00051.safetensors +3 -0
  28. model-00025-of-00051.safetensors +3 -0
  29. model-00026-of-00051.safetensors +3 -0
  30. model-00027-of-00051.safetensors +3 -0
  31. model-00028-of-00051.safetensors +3 -0
  32. model-00029-of-00051.safetensors +3 -0
  33. model-00030-of-00051.safetensors +3 -0
  34. model-00031-of-00051.safetensors +3 -0
  35. model-00032-of-00051.safetensors +3 -0
  36. model-00033-of-00051.safetensors +3 -0
  37. model-00034-of-00051.safetensors +3 -0
  38. model-00035-of-00051.safetensors +3 -0
  39. model-00036-of-00051.safetensors +3 -0
  40. model-00037-of-00051.safetensors +3 -0
  41. model-00038-of-00051.safetensors +3 -0
  42. model-00039-of-00051.safetensors +3 -0
  43. model-00040-of-00051.safetensors +3 -0
  44. model-00041-of-00051.safetensors +3 -0
  45. model-00042-of-00051.safetensors +3 -0
  46. model-00043-of-00051.safetensors +3 -0
  47. model-00044-of-00051.safetensors +3 -0
  48. model-00045-of-00051.safetensors +3 -0
  49. model-00046-of-00051.safetensors +3 -0
  50. model-00047-of-00051.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - TheDrummer/Behemoth-123B-v2.2
4
+ - anthracite-org/magnum-v4-123b
5
+ library_name: transformers
6
+ tags:
7
+ - mergekit
8
+ - merge
9
+
10
+ ---
11
+ # Behemoth-v2.2-Magnum-v4-123B
12
+
13
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
14
+
15
+ ## Merge Details
16
+ ### Merge Method
17
+
18
+ This model was merged using the SLERP merge method.
19
+
20
+ ### Models Merged
21
+
22
+ The following models were included in the merge:
23
+ * [TheDrummer/Behemoth-123B-v2.2](https://huggingface.co/TheDrummer/Behemoth-123B-v2.2)
24
+ * [anthracite-org/magnum-v4-123b](https://huggingface.co/anthracite-org/magnum-v4-123b)
25
+
26
+ ### Configuration
27
+
28
+ The following YAML configuration was used to produce this model:
29
+
30
+ ```yaml
31
+ models:
32
+ - model: TheDrummer/Behemoth-123B-v2.2
33
+ - model: anthracite-org/magnum-v4-123b
34
+ merge_method: slerp
35
+ base_model: TheDrummer/Behemoth-123B-v2.2
36
+ parameters:
37
+ t: [0.1, 0.3, 0.6, 0.3, 0.1]
38
+ dtype: float16
39
+
40
+ ```
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "TheDrummer/Behemoth-123B-v2.2",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 12288,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 28672,
14
+ "max_position_embeddings": 131072,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 96,
17
+ "num_hidden_layers": 88,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "torch_dtype": "float16",
24
+ "transformers_version": "4.46.3",
25
+ "use_cache": true,
26
+ "vocab_size": 32768
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ models:
2
+ - model: TheDrummer/Behemoth-123B-v2.2
3
+ - model: anthracite-org/magnum-v4-123b
4
+ merge_method: slerp
5
+ base_model: TheDrummer/Behemoth-123B-v2.2
6
+ parameters:
7
+ t: [0.1, 0.3, 0.6, 0.3, 0.1]
8
+ dtype: float16
model-00001-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af54ed31bee01e293fbe5354ba4db86017e3a680e6bab6e00d8218a9fa3187be
3
+ size 4378928488
model-00002-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c8bc6c0afdb2f4565da3c5bb2b603308359472e8c29cbe019fb5806bb91e29e2
3
+ size 4907411072
model-00003-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c1dfefc7ab8a125619a2ffca705c792f446f691533e8cdbf105c02c62ce9fa87
3
+ size 4806747888
model-00004-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c17a3e64e338b74bb6d81c5aec76cadcf84f36576ba2200ac38aa22dc8305a28
3
+ size 4831938528
model-00005-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:47ff003114a1be8fdc72a75d5c7ae9de2bd58494f27a6a8a19ce1d8a1a3cd240
3
+ size 4831938536
model-00006-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee4ee012ecf4c90fed4df69bfb8c7c8ce5ba4d5ab5553cc8ddf9a0228d29e0dc
3
+ size 4907411080
model-00007-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77c65fbdce702bfd8cdc9a6e85c02538eca20027956b5483205ecbe76451a9d1
3
+ size 4806747888
model-00008-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:37fbdb30de4869b94eb6ad6c1f260402e7b618468ae0d64164285210013acb79
3
+ size 4831938520
model-00009-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bab661a85b20a3b7a7fc20598809bfbcc8ec804564b4d690c10ee236dd0e0894
3
+ size 4831938536
model-00010-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8ced1a8a6c214938fdac3b956fe3b08426b661fbc88d1cf229bc2d2dc9130580
3
+ size 4907411080
model-00011-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0df578d0d8a1e5e8c6c2c37691cc2d2ad4d394ca112e2f7bcdf83f9aee56def7
3
+ size 4806747888
model-00012-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:038261e2dfc74bc19ff9c0dfd4569995eeb101b8bade2a7500b58da61d919b68
3
+ size 4831938528
model-00013-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0de14509d444753184caa74963ec43dc1361a142441c439f509683fc9a1fdc2
3
+ size 4831938536
model-00014-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0699b193a695cd4645484cb7da8b6d48b3e6cc794fb5220ffb502b6db2ec351
3
+ size 4907411072
model-00015-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c511f6a1f76576326e89ce8b2a4a92ee90302f9b5d42a261169639e9a694ed32
3
+ size 4806747888
model-00016-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5054680d28d9bc934276a8bac4858a136edcd36e7835953a2a7f79cddea914d2
3
+ size 4831938528
model-00017-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d3a80e293579a03c4e157bbf6666268624ee95c7d17e50b44fb3f7eb381f36d3
3
+ size 4831938536
model-00018-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7aaf6db6e906aef4224d40db365a05bef5bff7ca47773a1fb331ed31c6403ae9
3
+ size 4907411080
model-00019-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f9a0bd971f0af39960555766bde5ffe2f7ffd556f2b256ddbe5220a348e28459
3
+ size 4806747888
model-00020-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:91203b19b3e72a7880a42ff38328c8ac1e017b51d0bc95a53d9832053113be98
3
+ size 4831938528
model-00021-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:56c4fc3d41ff4520801eab5709012ef9a6641e674f4f4ec73dd1f11bf26357c4
3
+ size 4831938528
model-00022-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f4217f80d1582725ecfe4d218e86aebc2129553e852bd3d8fb7f854e36ebf78c
3
+ size 4907411080
model-00023-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36145014526f9cbcd3da96dc1efa43505842b13c736fb0501cfd4cb338c70692
3
+ size 4806747888
model-00024-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:392f67f0c68dfefef9a5a5b5cedb946bbc5428c5def2367b40f416a933176d6b
3
+ size 4831938528
model-00025-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d477017c01889e1bca101a2d2d09b4eb0ce1aad66f1b94172ca6ce92d745cd15
3
+ size 4831938536
model-00026-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:41c22b0e91f7125f407013726db5681b7dbb009ebe45a00e6889622fa7881d85
3
+ size 4907411080
model-00027-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0114220ea186fdd36f81026f4e5f98d1cee39877d29e9378256311b17f638d19
3
+ size 4806747880
model-00028-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9982cc883e37a8450817c2e9ffa268104b7b028cc61fbb9fdd5950ec79b99267
3
+ size 4831938528
model-00029-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3af6f16d635280f17c36a94f1f5ff3bbeb2a04477e58a1bc0c29367748e403a3
3
+ size 4831938536
model-00030-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6acf2dc350580cf25e062963cd8f9d53de079db05b6ab60c89fded847a488938
3
+ size 4907411080
model-00031-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:053bf98b78d9f241560ed6b48559d694e9a737b1a96cce0c01ef5667d53b8c66
3
+ size 4806747888
model-00032-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:918baf5b9a1cc095d7661b19091532ca4c1706801958047a9d8fa3abdf9f8cf0
3
+ size 4831938528
model-00033-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9ffc376fa538fcf108395674906aeab32396fe118558e9c9725abfaa80e14d6b
3
+ size 4831938528
model-00034-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:23e153fd3e9bcb0bab4ec2b34ca448efb42f9d35b7408c1b6ac19f2ef5a50bca
3
+ size 4907411080
model-00035-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1127ed7663854559296d9d7e91e7b34dfa1d52586e501f31297ae54e66776076
3
+ size 4806747888
model-00036-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a9df9d429f83f58eef602b5ce711705ceedc2f10bb3010288f40fb500824d590
3
+ size 4831938528
model-00037-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c7e9e9a85a54e95946cd7c5076003927706eba2e81251e4cf9bf55a5887c115
3
+ size 4831938536
model-00038-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ded014b3828f4f5ea8b92bd46f2e1ee8d04d65092bc2780264ed9b224a8cb684
3
+ size 4907411080
model-00039-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3955d0707a4b0fa34f6f499d02eeac5cd0ead0b5e27426bd71323aff0cf15a9
3
+ size 4806747888
model-00040-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:369704bfee0d3c388469fbd890800bc6a84c440214028ae3cbbbc9fb51adb697
3
+ size 4831938528
model-00041-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:43ad86b4a98ef9601c6d364e8d2a28bd25de24bfd46edece433657ad79b5cd0f
3
+ size 4831938536
model-00042-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ac5dd63f86c0dd40a6c1e7151b9422e1c5213c22dd6e33d131415ca9abeea90d
3
+ size 4907411080
model-00043-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d7b8c98d343971d34afb6b8b89a8a183b8a38b0c473a7805e7ac2e91bd363e25
3
+ size 4806747888
model-00044-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:20f4751ea474b730719f64253af3f797bdc77e84cfd3b6391600ce67fee64d58
3
+ size 4831938528
model-00045-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:61badd5603346443bc0dd77888b5da6a85742a93d6444c1b19dbc51a4b6a2a19
3
+ size 4831938536
model-00046-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af52a792de465588bf3473992fe4502f6fe298c3371fa983e33b4038513a9b56
3
+ size 4907411072
model-00047-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:40bffbbce3c0e00cbb027fd6ebb6ab2787da11f8f2d4ef84e08145b759a9e24d
3
+ size 4806747888