Hugging Face's logo

denru
/
Monstral-123B-v2-Behemoth-v2.2-Magnum-v4-123B-169B

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
denru commited on
Commit
e894c0d
·
verified ·
1 Parent(s): 401cccf

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +41 -0
  2. config.json +27 -0
  3. mergekit_config.yml +10 -0
  4. model-00001-of-00071.safetensors +3 -0
  5. model-00002-of-00071.safetensors +3 -0
  6. model-00003-of-00071.safetensors +3 -0
  7. model-00004-of-00071.safetensors +3 -0
  8. model-00005-of-00071.safetensors +3 -0
  9. model-00006-of-00071.safetensors +3 -0
  10. model-00007-of-00071.safetensors +3 -0
  11. model-00008-of-00071.safetensors +3 -0
  12. model-00009-of-00071.safetensors +3 -0
  13. model-00010-of-00071.safetensors +3 -0
  14. model-00011-of-00071.safetensors +3 -0
  15. model-00012-of-00071.safetensors +3 -0
  16. model-00013-of-00071.safetensors +3 -0
  17. model-00014-of-00071.safetensors +3 -0
  18. model-00015-of-00071.safetensors +3 -0
  19. model-00016-of-00071.safetensors +3 -0
  20. model-00017-of-00071.safetensors +3 -0
  21. model-00018-of-00071.safetensors +3 -0
  22. model-00019-of-00071.safetensors +3 -0
  23. model-00020-of-00071.safetensors +3 -0
  24. model-00021-of-00071.safetensors +3 -0
  25. model-00022-of-00071.safetensors +3 -0
  26. model-00023-of-00071.safetensors +3 -0
  27. model-00024-of-00071.safetensors +3 -0
  28. model-00025-of-00071.safetensors +3 -0
  29. model-00026-of-00071.safetensors +3 -0
  30. model-00027-of-00071.safetensors +3 -0
  31. model-00028-of-00071.safetensors +3 -0
  32. model-00029-of-00071.safetensors +3 -0
  33. model-00030-of-00071.safetensors +3 -0
  34. model-00031-of-00071.safetensors +3 -0
  35. model-00032-of-00071.safetensors +3 -0
  36. model-00033-of-00071.safetensors +3 -0
  37. model-00034-of-00071.safetensors +3 -0
  38. model-00035-of-00071.safetensors +3 -0
  39. model-00036-of-00071.safetensors +3 -0
  40. model-00037-of-00071.safetensors +3 -0
  41. model-00038-of-00071.safetensors +3 -0
  42. model-00039-of-00071.safetensors +3 -0
  43. model-00040-of-00071.safetensors +3 -0
  44. model-00041-of-00071.safetensors +3 -0
  45. model-00042-of-00071.safetensors +3 -0
  46. model-00043-of-00071.safetensors +3 -0
  47. model-00044-of-00071.safetensors +3 -0
  48. model-00045-of-00071.safetensors +3 -0
  49. model-00046-of-00071.safetensors +3 -0
  50. model-00047-of-00071.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - MidoriUnko/Behemoth-v2.2-Magnum-v4-123B
4
+ - MarsupialAI/Monstral-123B-v2
5
+ library_name: transformers
6
+ tags:
7
+ - mergekit
8
+ - merge
9
+
10
+ ---
11
+ # merged_model
12
+
13
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
14
+
15
+ ## Merge Details
16
+ ### Merge Method
17
+
18
+ This model was merged using the passthrough merge method.
19
+
20
+ ### Models Merged
21
+
22
+ The following models were included in the merge:
23
+ * [MidoriUnko/Behemoth-v2.2-Magnum-v4-123B](https://huggingface.co/MidoriUnko/Behemoth-v2.2-Magnum-v4-123B)
24
+ * [MarsupialAI/Monstral-123B-v2](https://huggingface.co/MarsupialAI/Monstral-123B-v2)
25
+
26
+ ### Configuration
27
+
28
+ The following YAML configuration was used to produce this model:
29
+
30
+ ```yaml
31
+ slices:
32
+ - sources:
33
+ - model: MarsupialAI/Monstral-123B-v2
34
+ layer_range: [0, 61]
35
+ - sources:
36
+ - model: MidoriUnko/Behemoth-v2.2-Magnum-v4-123B
37
+ layer_range: [27, 88]
38
+ merge_method: passthrough
39
+ dtype: float16
40
+ name: monstral
41
+ ```
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "MidoriUnko/Behemoth-v2.2-Magnum-v4-123B",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 12288,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 28672,
14
+ "max_position_embeddings": 131072,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 96,
17
+ "num_hidden_layers": 122,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "torch_dtype": "float16",
24
+ "transformers_version": "4.47.1",
25
+ "use_cache": true,
26
+ "vocab_size": 32768
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ slices:
2
+ - sources:
3
+ - model: MarsupialAI/Monstral-123B-v2
4
+ layer_range: [0, 61]
5
+ - sources:
6
+ - model: MidoriUnko/Behemoth-v2.2-Magnum-v4-123B
7
+ layer_range: [27, 88]
8
+ merge_method: passthrough
9
+ dtype: float16
10
+ name: monstral
model-00001-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:41ff74477d0081835b213aec3d7ecc35a21e7e53467c58566048c5392602261d
3
+ size 4378928488
model-00002-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e17a9b2a1e4500c5a9fd7a219e880a8670bf688b68dc9ab29618a0aec74e83b
3
+ size 4907411072
model-00003-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e1b6590c5b4e6d37323ce3d6666881e987e4a2321d3d9621820bb0319079defe
3
+ size 4806747888
model-00004-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:220eb35df35d5c24c31528ecbeb3570bc7ee464ccf4b1e50654e50dae2a719b9
3
+ size 4831938528
model-00005-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:063e5db6771b651c709190ef1d0ffef89dae2204648044ccaad30d258d23cc95
3
+ size 4831938536
model-00006-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d54039dd70f461b164c0905f9203f8aaf9bf0921da3ccb6a48003ec5da824f4
3
+ size 4907411080
model-00007-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0752ad241695d36a76a973463ffecba679e19c621863283f029c22f55481b225
3
+ size 4806747888
model-00008-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:168640080680cb0f534d8e89ccf2e51e24526147c5f387fe1e52f5f43119e789
3
+ size 4831938520
model-00009-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c28a21aecb702c8fc049b8ae20d4f57672f09bd392cd58e815d4a0efeabc56f
3
+ size 4831938536
model-00010-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5291e0cabd1baaa6b7871a64207c367e633817c95bb3bffe8f965b72474d7d50
3
+ size 4907411080
model-00011-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e902aa69bafc0553661efd2fbb12e6a8564a4bc33b30567680ca39a38e26c2b
3
+ size 4806747888
model-00012-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58cddae4428e5bf13d1c15bf39c976613ecaacb42c9a5f26a477e3fdd44f3be1
3
+ size 4831963216
model-00013-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:59955349d559bde405fce51dc53a35dfe47e68213255250f1e34395bee214c75
3
+ size 4831938536
model-00014-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6d529a46d9340ae00c0c4a36d4ef392a94dbad0374a1bd8ba513a2e2f638c441
3
+ size 4882220448
model-00015-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b7e1c3a946cb71f7e7bf0e876841faa8faa373272a518866f89d8da24b8aadab
3
+ size 4932601704
model-00016-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f2ade0c25940aaadf1ca57a8f6a888e162fbfb71dcc49b09b1020d95e80d2f10
3
+ size 4731275336
model-00017-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:90d1f62e5519192c2c186940fc4af6754af749ea0c1941c6703f701e442a9c29
3
+ size 4831938536
model-00018-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c99d5a679a1534de4400b4bb6845f04d271a2add9b27839205dcfa8c55781a64
3
+ size 4882220448
model-00019-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8d22ad9793ddd04711c57c8d42116a883b74cc8324a62a800f3afff0ced1154a
3
+ size 4932601704
model-00020-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2434ef85595f843bfc6f185da916e5f2b83483815370181945554096561b6685
3
+ size 4781557256
model-00021-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6bbc51e614472fbf0b9368bc02021394b3b59298eaf50a10feaf954c5254dc39
3
+ size 4831938536
model-00022-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f9f5016c6ed0c9e0ddf4292c6c927e2d64261a006501bf04f39366f3ccb617ea
3
+ size 4831938528
model-00023-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e7af9048d9bab870396d90c085f2af2683259ca83b61323381d3226dcded70b5
3
+ size 4831938536
model-00024-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1d4af440d36d9235362219d523968edf54650d4e7bb384699c0be6db58f93de6
3
+ size 4831938536
model-00025-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:82c84d77e159c8b0d377fee44656b148ba16341082447289cb98745603b6ef97
3
+ size 4831938536
model-00026-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de8eb365231249dde44260095d69769a85496aaa78541e53111c90341332d810
3
+ size 4882220448
model-00027-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c83d031ba86c00b5326e4b219e0e33b27e9cbc519abfdee678e704f1f74df4cc
3
+ size 4932601704
model-00028-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c8047dcd4899685d47a593d1c48c196a8e2af8108405b7f83991478dba53e72d
3
+ size 4731275336
model-00029-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3dc036e5ba00c401285f0aaa699cf2ba548d8de59050d2bd39e75cb516becb58
3
+ size 4831938536
model-00030-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ab5aacf087ea755c13f12af95ff66dbe855dd6943dec437a31f1c872f37ba240
3
+ size 4882220448
model-00031-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c11670db4981aaf8deb379d8edacbc8287efb384831175c097ef52907cf0449
3
+ size 4932601704
model-00032-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:037e40855f176417cc9429187695832bd2747a98c9d42b25fd9bd36ab8e25821
3
+ size 4781557256
model-00033-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0ca8314ce616244fd3a1c0856d7bb50829d89522c44e75efdca792d81b4205f8
3
+ size 4831938536
model-00034-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:df730ec37a364ea2ad7cbbec8ec5ba0bf61e9965be75932f0978cc5481e15e6f
3
+ size 4831938528
model-00035-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0e010f1697cbe58eacb8ff4d6d968efdeb21da11dc10966c9f635b8f377418bc
3
+ size 4831938536
model-00036-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1508529063dac3c48db4ff8061f5f90c865bd5b2b31cc5a980a44d6aeeaaacfb
3
+ size 4831938536
model-00037-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3961df4fb44f835581e922ef688ac5e46f27580b45f8d83b511057b0b6a5ae44
3
+ size 4831938536
model-00038-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e5c4941aaa5a8f1b2a2bcb154c99b1e7bd3a42307014a38bcfb80e5891058f9
3
+ size 4882220448
model-00039-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58e516c98e891f316bec3a9600f8c011c0b3b0af0c5d351da57bbf5ea1ed645d
3
+ size 4932601704
model-00040-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a23ee3ce538e7af7b7b7394245364118dd4b44c9b52c1ef96d3d66c6a5420548
3
+ size 4731275336
model-00041-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:127a71e61b7664091c97f4599314beb8c7419ee2bf68b20e49d285d85a423570
3
+ size 4831938536
model-00042-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c73700ec1285bdf56632325d68eacebd85275fd7fb32853844ed3e1c942fa681
3
+ size 4882220448
model-00043-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc88155924be1f6d868d9e11b012c177d6483970439e0f6345d47c7338765157
3
+ size 4932601704
model-00044-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b92f4010ac2310400bd8db84cf32ee4d96015036ae9a633910abb950d4dcea74
3
+ size 4781557256
model-00045-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60aaba8220c984f3082490b4fc88c1f7daf5145498d12a574ada779d61b3d3c2
3
+ size 4831938536
model-00046-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a6ef13c021190527262bff43170e2b88209742c9f111ae278e30b86230e015a0
3
+ size 4831938528
model-00047-of-00071.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36b5eb72fff0a1d52d463fcf841b58aca747c8351ad8d3fcf94b709653592e65
3
+ size 4831938536