README.md

---
base_model:
- tarotscientist/llama-2-7b-tarotreader
- Nitral-Archive/HerculeanSea-7b-128k
- Norquinal/Mistral-7B-storywriter
- teknium/llama-deus-7b-v3-lora-merged
- Undi95/BigL-7B
- Undi95/MistRP-Dolphin-7B
- Undi95/Toppy-M-7B
- NousResearch/Yarn-Mistral-7b-128k
- Undi95/Mistral-ClaudeLimaRP-v3-7B
- teknium/Mistral-Trismegistus-7B
- alexandrabenamar/Mistral-7B-Instruct-v0.2-Magic
- CallComply/zephyr-7b-beta-128k
- teknium/Hermes-Trismegistus-Mistral-7B
- Undi95/LewdMistral-7B-0.2
library_name: transformers
tags:
- mergekit
- merge

---
# merge

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

## Merge Details
### Merge Method

This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [NousResearch/Yarn-Mistral-7b-128k](https://huggingface.co/NousResearch/Yarn-Mistral-7b-128k) as a base.

### Models Merged

The following models were included in the merge:
* [tarotscientist/llama-2-7b-tarotreader](https://huggingface.co/tarotscientist/llama-2-7b-tarotreader)
* [Nitral-Archive/HerculeanSea-7b-128k](https://huggingface.co/Nitral-Archive/HerculeanSea-7b-128k)
* [Norquinal/Mistral-7B-storywriter](https://huggingface.co/Norquinal/Mistral-7B-storywriter)
* [teknium/llama-deus-7b-v3-lora-merged](https://huggingface.co/teknium/llama-deus-7b-v3-lora-merged)
* [Undi95/BigL-7B](https://huggingface.co/Undi95/BigL-7B)
* [Undi95/MistRP-Dolphin-7B](https://huggingface.co/Undi95/MistRP-Dolphin-7B)
* [Undi95/Toppy-M-7B](https://huggingface.co/Undi95/Toppy-M-7B)
* [Undi95/Mistral-ClaudeLimaRP-v3-7B](https://huggingface.co/Undi95/Mistral-ClaudeLimaRP-v3-7B)
* [teknium/Mistral-Trismegistus-7B](https://huggingface.co/teknium/Mistral-Trismegistus-7B)
* [alexandrabenamar/Mistral-7B-Instruct-v0.2-Magic](https://huggingface.co/alexandrabenamar/Mistral-7B-Instruct-v0.2-Magic)
* [CallComply/zephyr-7b-beta-128k](https://huggingface.co/CallComply/zephyr-7b-beta-128k)
* [teknium/Hermes-Trismegistus-Mistral-7B](https://huggingface.co/teknium/Hermes-Trismegistus-Mistral-7B)
* [Undi95/LewdMistral-7B-0.2](https://huggingface.co/Undi95/LewdMistral-7B-0.2)

### Configuration

The following YAML configuration was used to produce this model:

```yaml
### This the config.yml for ABC_Books/test001 ###
 models:

  ### Models that contribute a large 128K context window ###
  - model: CallComply/zephyr-7b-beta-128k
    parameters:
      weight: 0.1154
      density: 0.9
  - model: Nitral-Archive/HerculeanSea-7b-128k
    parameters:
      weight: 0.1154
      density: 0.9
  - model: NousResearch/Yarn-Mistral-7b-128k
    parameters:
      weight: 0.1154
      density: 0.9

  ### Models with finetune training on occult knownledge ###
  - model: teknium/llama-deus-7b-v3-lora-merged
    parameters:
      weight: 0.0769
      density: 0.9
  - model: teknium/Hermes-Trismegistus-Mistral-7B
    parameters:
      weight: 0.0769
      density: 0.9
  - model: alexandrabenamar/Mistral-7B-Instruct-v0.2-Magic
    parameters:
      weight: 0.0769
      density: 0.9
  - model: tarotscientist/llama-2-7b-tarotreader
    parameters:
      weight: 0.0769
      density: 0.9
  - model: teknium/Mistral-Trismegistus-7B
    parameters:
      weight: 0.0769
      density: 0.9

  ### Talkative model with a large context window ###
  - model: Norquinal/Mistral-7B-storywriter
    parameters:
      weight: 0.0769
      density: 0.9

  ### Models with finetune training to be uncensored use some crass diction ###
  - model: Undi95/BigL-7B
    parameters:
      weight: 0.0384
      density: 0.9
  - model: Undi95/LewdMistral-7B-0.2
    parameters:
      weight: 0.0385
      density: 0.9
  - model: Undi95/MistRP-Dolphin-7B
    parameters:
      weight: 0.0385
      density: 0.9
  - model: Undi95/Mistral-ClaudeLimaRP-v3-7B
    parameters:
      weight: 0.0385
      density: 0.9
  - model: Undi95/Toppy-M-7B
    parameters:
      weight: 0.0385
      density: 0.9

  ### The use of DARES has been shown to “Densify” standard model lending to a more robust model when paired with a high “density:” numbers ###
 merge_method: dare_ties

  ### This model best exemplifies the closest match to all of the features needed in the final model ###
 base_model: NousResearch/Yarn-Mistral-7b-128k
 parameters:

  ### When “densifing” models the model size tends to grow without normalize
  normalize: true
  int8_mask: true
 dtype: float16

```