Hugging Face's logo

MrRobotoAI
/
Test001a

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
custom_code
text-generation-inference
Inference Endpoints
Model card Files Community
Test001a
File size: 4,475 Bytes 
b0e3382
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
 
---
base_model:
- tarotscientist/llama-2-7b-tarotreader
- Nitral-Archive/HerculeanSea-7b-128k
- Norquinal/Mistral-7B-storywriter
- teknium/llama-deus-7b-v3-lora-merged
- Undi95/BigL-7B
- Undi95/MistRP-Dolphin-7B
- Undi95/Toppy-M-7B
- NousResearch/Yarn-Mistral-7b-128k
- Undi95/Mistral-ClaudeLimaRP-v3-7B
- teknium/Mistral-Trismegistus-7B
- alexandrabenamar/Mistral-7B-Instruct-v0.2-Magic
- CallComply/zephyr-7b-beta-128k
- teknium/Hermes-Trismegistus-Mistral-7B
- Undi95/LewdMistral-7B-0.2
library_name: transformers
tags:
- mergekit
- merge

---
# merge

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

## Merge Details
### Merge Method

This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [NousResearch/Yarn-Mistral-7b-128k](https://huggingface.co/NousResearch/Yarn-Mistral-7b-128k) as a base.

### Models Merged

The following models were included in the merge:
* [tarotscientist/llama-2-7b-tarotreader](https://huggingface.co/tarotscientist/llama-2-7b-tarotreader)
* [Nitral-Archive/HerculeanSea-7b-128k](https://huggingface.co/Nitral-Archive/HerculeanSea-7b-128k)
* [Norquinal/Mistral-7B-storywriter](https://huggingface.co/Norquinal/Mistral-7B-storywriter)
* [teknium/llama-deus-7b-v3-lora-merged](https://huggingface.co/teknium/llama-deus-7b-v3-lora-merged)
* [Undi95/BigL-7B](https://huggingface.co/Undi95/BigL-7B)
* [Undi95/MistRP-Dolphin-7B](https://huggingface.co/Undi95/MistRP-Dolphin-7B)
* [Undi95/Toppy-M-7B](https://huggingface.co/Undi95/Toppy-M-7B)
* [Undi95/Mistral-ClaudeLimaRP-v3-7B](https://huggingface.co/Undi95/Mistral-ClaudeLimaRP-v3-7B)
* [teknium/Mistral-Trismegistus-7B](https://huggingface.co/teknium/Mistral-Trismegistus-7B)
* [alexandrabenamar/Mistral-7B-Instruct-v0.2-Magic](https://huggingface.co/alexandrabenamar/Mistral-7B-Instruct-v0.2-Magic)
* [CallComply/zephyr-7b-beta-128k](https://huggingface.co/CallComply/zephyr-7b-beta-128k)
* [teknium/Hermes-Trismegistus-Mistral-7B](https://huggingface.co/teknium/Hermes-Trismegistus-Mistral-7B)
* [Undi95/LewdMistral-7B-0.2](https://huggingface.co/Undi95/LewdMistral-7B-0.2)

### Configuration

The following YAML configuration was used to produce this model:

```yaml
### This the config.yml for ABC_Books/test001 ###
 models:

  ### Models that contribute a large 128K context window ###
  - model: CallComply/zephyr-7b-beta-128k
    parameters:
      weight: 0.1154
      density: 0.9
  - model: Nitral-Archive/HerculeanSea-7b-128k
    parameters:
      weight: 0.1154
      density: 0.9
  - model: NousResearch/Yarn-Mistral-7b-128k
    parameters:
      weight: 0.1154
      density: 0.9

  ### Models with finetune training on occult knownledge ###
  - model: teknium/llama-deus-7b-v3-lora-merged
    parameters:
      weight: 0.0769
      density: 0.9
  - model: teknium/Hermes-Trismegistus-Mistral-7B
    parameters:
      weight: 0.0769
      density: 0.9
  - model: alexandrabenamar/Mistral-7B-Instruct-v0.2-Magic
    parameters:
      weight: 0.0769
      density: 0.9
  - model: tarotscientist/llama-2-7b-tarotreader
    parameters:
      weight: 0.0769
      density: 0.9
  - model: teknium/Mistral-Trismegistus-7B
    parameters:
      weight: 0.0769
      density: 0.9

  ### Talkative model with a large context window ###
  - model: Norquinal/Mistral-7B-storywriter
    parameters:
      weight: 0.0769
      density: 0.9

  ### Models with finetune training to be uncensored use some crass diction ###
  - model: Undi95/BigL-7B
    parameters:
      weight: 0.0384
      density: 0.9
  - model: Undi95/LewdMistral-7B-0.2
    parameters:
      weight: 0.0385
      density: 0.9
  - model: Undi95/MistRP-Dolphin-7B
    parameters:
      weight: 0.0385
      density: 0.9
  - model: Undi95/Mistral-ClaudeLimaRP-v3-7B
    parameters:
      weight: 0.0385
      density: 0.9
  - model: Undi95/Toppy-M-7B
    parameters:
      weight: 0.0385
      density: 0.9

  ### The use of DARES has been shown to “Densify” standard model lending to a more robust model when paired with a high “density:” numbers ###
 merge_method: dare_ties

  ### This model best exemplifies the closest match to all of the features needed in the final model ###
 base_model: NousResearch/Yarn-Mistral-7b-128k
 parameters:

  ### When “densifing” models the model size tends to grow without normalize
  normalize: true
  int8_mask: true
 dtype: float16

```