Hugging Face's logo

knifeayumu
/
BeMag-123B-v0.1

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Model card Files Community
1
knifeayumu commited on
Commit
de7b3e7
·
verified ·
1 Parent(s): 3c04b62

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +45 -0
  2. config.json +27 -0
  3. mergekit_config.yml +13 -0
  4. model-00001-of-00051.safetensors +3 -0
  5. model-00002-of-00051.safetensors +3 -0
  6. model-00003-of-00051.safetensors +3 -0
  7. model-00004-of-00051.safetensors +3 -0
  8. model-00005-of-00051.safetensors +3 -0
  9. model-00006-of-00051.safetensors +3 -0
  10. model-00007-of-00051.safetensors +3 -0
  11. model-00008-of-00051.safetensors +3 -0
  12. model-00009-of-00051.safetensors +3 -0
  13. model-00010-of-00051.safetensors +3 -0
  14. model-00011-of-00051.safetensors +3 -0
  15. model-00012-of-00051.safetensors +3 -0
  16. model-00013-of-00051.safetensors +3 -0
  17. model-00014-of-00051.safetensors +3 -0
  18. model-00015-of-00051.safetensors +3 -0
  19. model-00016-of-00051.safetensors +3 -0
  20. model-00017-of-00051.safetensors +3 -0
  21. model-00018-of-00051.safetensors +3 -0
  22. model-00019-of-00051.safetensors +3 -0
  23. model-00020-of-00051.safetensors +3 -0
  24. model-00021-of-00051.safetensors +3 -0
  25. model-00022-of-00051.safetensors +3 -0
  26. model-00023-of-00051.safetensors +3 -0
  27. model-00024-of-00051.safetensors +3 -0
  28. model-00025-of-00051.safetensors +3 -0
  29. model-00026-of-00051.safetensors +3 -0
  30. model-00027-of-00051.safetensors +3 -0
  31. model-00028-of-00051.safetensors +3 -0
  32. model-00029-of-00051.safetensors +3 -0
  33. model-00030-of-00051.safetensors +3 -0
  34. model-00031-of-00051.safetensors +3 -0
  35. model-00032-of-00051.safetensors +3 -0
  36. model-00033-of-00051.safetensors +3 -0
  37. model-00034-of-00051.safetensors +3 -0
  38. model-00035-of-00051.safetensors +3 -0
  39. model-00036-of-00051.safetensors +3 -0
  40. model-00037-of-00051.safetensors +3 -0
  41. model-00038-of-00051.safetensors +3 -0
  42. model-00039-of-00051.safetensors +3 -0
  43. model-00040-of-00051.safetensors +3 -0
  44. model-00041-of-00051.safetensors +3 -0
  45. model-00042-of-00051.safetensors +3 -0
  46. model-00043-of-00051.safetensors +3 -0
  47. model-00044-of-00051.safetensors +3 -0
  48. model-00045-of-00051.safetensors +3 -0
  49. model-00046-of-00051.safetensors +3 -0
  50. model-00047-of-00051.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - TheDrummer/Behemoth-123B-v1
4
+ - TheDrummer/Behemoth-123B-v1.1
5
+ - anthracite-org/magnum-v4-123b
6
+ library_name: transformers
7
+ tags:
8
+ - mergekit
9
+ - merge
10
+
11
+ ---
12
+ # merge
13
+
14
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
+
16
+ ## Merge Details
17
+ ### Merge Method
18
+
19
+ This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using [TheDrummer/Behemoth-123B-v1](https://huggingface.co/TheDrummer/Behemoth-123B-v1) as a base.
20
+
21
+ ### Models Merged
22
+
23
+ The following models were included in the merge:
24
+ * [TheDrummer/Behemoth-123B-v1.1](https://huggingface.co/TheDrummer/Behemoth-123B-v1.1)
25
+ * [anthracite-org/magnum-v4-123b](https://huggingface.co/anthracite-org/magnum-v4-123b)
26
+
27
+ ### Configuration
28
+
29
+ The following YAML configuration was used to produce this model:
30
+
31
+ ```yaml
32
+ models:
33
+ - model: TheDrummer/Behemoth-123B-v1
34
+ parameters:
35
+ weight: 0.425
36
+ - model: TheDrummer/Behemoth-123B-v1.1
37
+ parameters:
38
+ weight: 0.425
39
+ - model: anthracite-org/magnum-v4-123b
40
+ parameters:
41
+ weight: 0.15
42
+ merge_method: task_arithmetic
43
+ base_model: TheDrummer/Behemoth-123B-v1
44
+ dtype: bfloat16
45
+ ```
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "TheDrummer/Behemoth-123B-v1",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 12288,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 28672,
14
+ "max_position_embeddings": 131072,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 96,
17
+ "num_hidden_layers": 88,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "torch_dtype": "bfloat16",
24
+ "transformers_version": "4.46.2",
25
+ "use_cache": true,
26
+ "vocab_size": 32768
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ models:
2
+ - model: TheDrummer/Behemoth-123B-v1
3
+ parameters:
4
+ weight: 0.425
5
+ - model: TheDrummer/Behemoth-123B-v1.1
6
+ parameters:
7
+ weight: 0.425
8
+ - model: anthracite-org/magnum-v4-123b
9
+ parameters:
10
+ weight: 0.15
11
+ merge_method: task_arithmetic
12
+ base_model: TheDrummer/Behemoth-123B-v1
13
+ dtype: bfloat16
model-00001-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a67ba8bfa353d6cc9c45c8b900773e7f7c2313ccf36c78d432952b6e589a9b64
3
+ size 4378928504
model-00002-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aa0896f26d30b93e7085225e27eca57ce4dbd4ed5d57c2193cd23108e0bb51d9
3
+ size 4907411088
model-00003-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe127a25e4697fcf8579ed06154c7bb484f783e1f729cebd1cf07dee9fe2505d
3
+ size 4806747904
model-00004-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b7f6e583cf7606ef6490c1312ee4fd7cfb9ba5e1887f7432a63a0f8ae54f31cb
3
+ size 4831938544
model-00005-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6644d92f0fdd984a3809f3b35dd2be367caa241230ea79ef9baece2f6851a203
3
+ size 4831938552
model-00006-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bb6ca6566823cf3c831f0d2546ca168d666feea594806d120d95c70ac5d05fa6
3
+ size 4907411096
model-00007-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:508324fc88eb30b0a964c34b8a05feb734f9f697e9c103062ac67d85b127d8ef
3
+ size 4806747904
model-00008-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:83668f27b1f4508825fb4c46fa555e1b1d1c28612af18dfec410cc338984f9aa
3
+ size 4831938536
model-00009-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c5e8afd21d49366321ea7db67e5ee98588a28cf73c948526be9e57f2c453b854
3
+ size 4831938552
model-00010-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:45cc042e0a4ce5298be940de2cc1e525b407cb48c777db4bf557888b5fb0a741
3
+ size 4907411096
model-00011-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:44d8b4927d3919be49a64aba54f0ed289a8c41fda3f2e319b21be156c1d6986b
3
+ size 4806747904
model-00012-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b336c00fb77b1c8662e966e94dd1abeb0270f54d849e6110ace6a1c01c79b633
3
+ size 4831938544
model-00013-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3e95f6fba43e07f662b1b0f005d25b2ac3e3ae2f71ecf6788ce540f0566319a7
3
+ size 4831938552
model-00014-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8e6e616f4ecd90b79537422248d81c0566421dac4130921df5ba0ec70840564e
3
+ size 4907411088
model-00015-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0319684a59391741e58af65dffeefa3eca77590c49135aefc9e61e51eefc9722
3
+ size 4806747904
model-00016-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a6b0e727390408e37606b15e285d923a2f9aca21e8ded2be2d53b29467e3172
3
+ size 4831938544
model-00017-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:83e68cbeb69eb211ffc448eb0454a7f8ee982d7f7bb6fa4d4e5d4ee74068d863
3
+ size 4831938552
model-00018-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e68499553aa7ad76d9a82f30ecf09634ba609b5903d0e0a04f90187411aa21d
3
+ size 4907411096
model-00019-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:64dbf3ad285f8a4d4b22caa3864afc6f756a870d7a4e1087ddb5fe09006f8aea
3
+ size 4806747904
model-00020-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1acc267fd2e0617d3ab92ed42305f8adcc914b8dd8a1a7542beff0acb447a2c9
3
+ size 4831938544
model-00021-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5782eb731aef2ca052c3e56a2b7fcf89387270847338baeea3df9919816bf73e
3
+ size 4831938544
model-00022-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7ba3a7c5d857316f5cc48a0e57b7008e957c2ad99b2e209243c7bd8ef2e7fa16
3
+ size 4907411096
model-00023-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63f920c2061c38046acb12dc6fe64a7f7a332a7912cdd0677c699ac4f8103b1c
3
+ size 4806747904
model-00024-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2dee0e185398752a8320338871f202e924e90c127e6cfb28ffaef2249b986f77
3
+ size 4831938544
model-00025-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:35b868b4b5148b2844f71baeda353b0e0c04787bfa59e30ab4af634274075ab3
3
+ size 4831938552
model-00026-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ecd135b2f1124491843291f9ffa181ab5703cacb7102ac85b553ebb6be9d0def
3
+ size 4907411096
model-00027-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e0b91040b7a6a1275368190a25c8c040618f6061686de3619936c1339420f40a
3
+ size 4806747896
model-00028-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a343d35fb25a6050d73ebc33859cf23c8ce52648cac2120b6ec6ff85cba26a7
3
+ size 4831938544
model-00029-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0db38b7e39cfafb6671e5995711d9a6b51b67fb1eec5b0ef6927a5fd59a75a46
3
+ size 4831938552
model-00030-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9cc25da7cc393667ed33fc3bbccb8cc88c738ee8ea83f83026f56a652f248f9c
3
+ size 4907411096
model-00031-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9f4db1028b9db43ce73ffc74cf427b2b2f152eff5853a3e5b2089f1ad546f790
3
+ size 4806747904
model-00032-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8cea78249e2577ff6d5be6a1db0b6e4315e62f07e5301359935e1a54a92dd2b
3
+ size 4831938544
model-00033-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c85d49727fa8e41fc85eec81b8122b69ea78b755a259b2d92967e0fafd17f41
3
+ size 4831938544
model-00034-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:69620c4702cdd686c47e6f9a276d40b6eac1c00acc8d1b1858ec793f9f230c3f
3
+ size 4907411096
model-00035-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:57dbe12b3efcb7ac97a9ab0e12debc2150fa80c9be127cba845d177eb7304c36
3
+ size 4806747904
model-00036-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:488c556a8a1ab74c9260803fe26b1791ccc951ee360f9dd733b1c35b2917b476
3
+ size 4831938544
model-00037-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c7b728ec623c00974f1baad83e5e45407a8f838bd3a47dafdc999dd9bb1e9292
3
+ size 4831938552
model-00038-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:520e4f5f7ab10127d0fb753ec74af9c0a22626411cfce9bc161f46608d5f16ce
3
+ size 4907411096
model-00039-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eb68070704ca5528dc1056cb4f6a16ad03268b61c8f496e144d10f1c255aa652
3
+ size 4806747904
model-00040-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58a33506325e39d68a9d649b8537ef838662928c28c29cf6525147ac3cd22b2a
3
+ size 4831938544
model-00041-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de63feabc6b1d82ae4d4d0940270a6ed0801bc60e6a1f27fd64c79f2cad70636
3
+ size 4831938552
model-00042-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:62ee4d262cd1ca6370bceeb314c5a6a46a40ee6bcb2cc14ac83bbff2efd3955f
3
+ size 4907411096
model-00043-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ba73fb3a66ee9f44b0e52f2f775881bcf6eddfe452935b1fbfe2d266f9cc9f2c
3
+ size 4806747904
model-00044-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2cb32353f3fe7797138958733d574283cf7167a749a37cec126091902db06bcc
3
+ size 4831938544
model-00045-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0aaaa6780d05aa959d4c76a4716faca0d426c03b266c5d5f324bc7bc46601f5
3
+ size 4831938552
model-00046-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4eabe189960c6c8a49f2d15255ea4ad1dbebce0780f9ebe46a2ef50e26cf5cfe
3
+ size 4907411088
model-00047-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6926ca2aafc6b9f3c8df89090236615227cfee835711de49a81820fedf8aa617
3
+ size 4806747904