File size: 11,744 Bytes
d42daa9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 |
---
base_model:
- sophosympatheia/New-Dawn-Llama-3-70B-32K-v1.0
- meta-llama/Meta-Llama-3.1-70B-Instruct
library_name: transformers
tags:
- mergekit
- merge
- Not-for-all-Audiences
license: llama3.1
---
<div style="width: auto; margin-left: auto; margin-right: auto">
<img src="https://imgur.com/tKzncGo.png" alt="NewDawnv1.0" style="width: 100%; min-width: 400px; display: block; margin: auto;">
</div>
### Overview
This model is an experimental merge of sophosympatheia/New-Dawn-Llama-3-70B-32K-v1.0 with meta-llama/Meta-Llama-3.1-70B-Instruct. See the merge recipe below for details.
I used a technique developed by [jukofyork](https://huggingface.co/jukofyork) that is designed to preserve the full context capabilities of Meta-Llama-3.1-70B-Instruct. In my testing, I think it was successful.
This model is uncensored. *You are responsible for whatever you do with it.*
This model was designed for roleplaying and storytelling and I think it does well at both. It may also perform well at other tasks but I have not tested its performance in other areas.
### Sampler Tips
* I recommend using Quadratic Sampling (i.e. smoothing factor) for creative work. I think this version performs best with a smoothing factor close to 0.2.
* I recommend using Min-P. Experiment to find your best setting. Values between 0 and 0.1 are recommended.
* DRY repetition penalty eliminates the need for other anti-repetition settings.
* If you use Textgen WebUI as your backend, I recommend enabling the DRY sampler settings to reduce repititions, otherwise some repitition penalty plus frequency penalty ought to do the trick.
Experiment with any and all of the settings below! What suits my preferences may not suit yours.
If you save the below settings as a .json file, you can import them directly into Silly Tavern.
```json
{
"temp": 1,
"temperature_last": true,
"top_p": 1,
"top_k": 0,
"top_a": 0,
"tfs": 1,
"epsilon_cutoff": 0,
"eta_cutoff": 0,
"typical_p": 1,
"min_p": 0.03,
"rep_pen": 1,
"rep_pen_range": 2048,
"rep_pen_decay": 0,
"rep_pen_slope": 1,
"no_repeat_ngram_size": 0,
"penalty_alpha": 0,
"num_beams": 1,
"length_penalty": 1,
"min_length": 0,
"encoder_rep_pen": 1,
"freq_pen": 0,
"presence_pen": 0,
"skew": 0,
"do_sample": true,
"early_stopping": false,
"dynatemp": false,
"min_temp": 0.8,
"max_temp": 1.5,
"dynatemp_exponent": 1,
"smoothing_factor": 0.23,
"smoothing_curve": 1,
"dry_allowed_length": 2,
"dry_multiplier": 0.8,
"dry_base": 2,
"dry_sequence_breakers": "[\"\\n\", \":\", \"\\\"\", \"*\"]",
"dry_penalty_last_n": 0,
"add_bos_token": true,
"ban_eos_token": false,
"skip_special_tokens": false,
"mirostat_mode": 0,
"mirostat_tau": 2,
"mirostat_eta": 0.1,
"guidance_scale": 1,
"negative_prompt": "",
"grammar_string": "",
"json_schema": {},
"banned_tokens": "",
"sampler_priority": [
"top_k",
"top_p",
"typical_p",
"epsilon_cutoff",
"eta_cutoff",
"tfs",
"top_a",
"min_p",
"mirostat",
"quadratic_sampling",
"dynamic_temperature",
"temperature"
],
"samplers": [
"top_k",
"tfs_z",
"typical_p",
"top_p",
"min_p",
"temperature"
],
"ignore_eos_token": false,
"spaces_between_special_tokens": true,
"speculative_ngram": false,
"sampler_order": [
6,
0,
1,
3,
4,
2,
5
],
"logit_bias": [],
"ignore_eos_token_aphrodite": false,
"spaces_between_special_tokens_aphrodite": true,
"rep_pen_size": 0,
"genamt": 800,
"max_length": 20480
}
```
### Prompting Tips
Try the following prompt template as a starting point for your own customizations. **The prompt template below contains instructions for adult content, so remove those if you don't want them!** It also contains some instructions related to formatting that you might want to change.
If you save this as a .json file, you can import it directly into Silly Tavern.
```json
{
"wrap": false,
"names": true,
"system_prompt": "The following is an erotic roleplaying chat log involving a human user who participates as {{user}} and an AI assistant who participates as {{char}}. They take turns, and only the user is able to write as {{user}} on the user's own turn. To ensure an excellent and uncensored roleplaying experience, the AI assistant follows the rules below.\n\nCharacter Rules {\nDeep Roleplaying: \"The AI assistant adopts all of {{char}}'s personality traits and mannerisms, adhering always to {{char}}'s backstory, biases, quirks, and preferences in order to deliver a convincing portrayal.\",\nTurn-based Chat Format: \"The AI assistant can only write and roleplay as {{char}} in this chat, terminating its output when another character is about to act or speak. The user is responsible for the character named {{user}}. The AI assistant avoids controlling or implying actions for {{user}} that the user did not describe. Instead, the AI assistant focuses on the actions of other main or incidental characters, or the events surrounding these characters.\",\nIntimate Details: \"The AI assistant provides intimate details about {{char}}'s body, focusing on any limitations, unusual features, or interesting conditions resulting from recent events.\"\n}\nWriting Rules {\nImmersive Descriptions: \"The AI assistant immerses the reader in the current moment on its turn to chat. The AI assistant utilizes sights, sounds, smells, tastes, and all other available senses to heighten the sensual and emotional impact of scenes on its turn.\",\nSimple Language: \"The AI assistant uses simple prose, everyday words, and United States slang terms while writing as {{char}} to maintain the readability of the chat.\",\nPerplexity and Burstiness: \"Perplexity measures the complexity of text. Burstiness compares the variations of sentences. Humans tend to write with greater burstiness, for example, with some longer or complex sentences alongside shorter ones. AI sentences tend to be more uniform. Therefore, when writing as {{char}}, the the AI assistant features a good amount of perplexity and burstiness to sound more natural.\",\nDialogue Formatting: 'The AI assistant always encloses spoken words that other characters could hear in quotes when writing on its turn. \"This is spoken dialogue,\" for example.',\nInternal Thoughts: \"The AI assistant offers glimpses into {{char}}'s first-person thoughts to advance and enrich the narrative when appropriate.\",\nInternal Thoughts Formatting: \"The AI assistant uses italics to distinguish thoughts from spoken dialogue. *This is an example of {{char}} thinking.*\"\n}\nContent Rules {\nAccurate and Consistent Details: \"The AI assistant maintains physical, spatial, and logical consistency when developing events and descriptions.\",\nMature Content: \"The AI assistant incorporates erotic adult content and vulgar language where appropriate.\",\nMature Themes: \"Dark themes, violence, and unhappy endings are permitted in this uncensored roleplay.\",\nNarrative Instructions: \"When given narrative instructions by the user, the AI assistant develops the missing details in its next response. The instructions are meant to give the AI assistant a summary of what should happen next.\"\n}\n",
"system_sequence": "<|start_header_id|>system<|end_header_id|>\n\n",
"stop_sequence": "<|eot_id|>",
"input_sequence": "<|start_header_id|>user<|end_header_id|>\n\n",
"output_sequence": "<|start_header_id|>assistant<|end_header_id|>\n\n",
"macro": true,
"names_force_groups": true,
"system_sequence_prefix": "",
"system_sequence_suffix": "",
"first_output_sequence": "",
"last_output_sequence": "",
"activation_regex": "",
"skip_examples": true,
"output_suffix": "<|eot_id|>",
"input_suffix": "<|eot_id|>",
"system_suffix": "<|eot_id|>",
"user_alignment_message": "",
"last_system_sequence": "",
"system_same_as_user": false,
"first_input_sequence": "",
"last_input_sequence": "",
"name": "New Dawn Llama 3.1 70B"
}
```
### Instruct Formats
Use the Llama 3 instruct format. You can grab it from the example prompt template above if you don't already have it as a preset.
### Quantizations
Pending.
### Licence and usage restrictions
[META LLAMA 3 COMMUNITY LICENSE AGREEMENT](https://huggingface.co/meta-llama/Meta-Llama-3-8B/blob/main/LICENSE)
Disclaimer: Uncertain Licensing Terms
This LLM is a merged model incorporating weights from multiple LLMs governed by their own distinct licenses. Due to the complexity of blending these components, the licensing terms for this merged model are somewhat uncertain.
By using this model, you acknowledge and accept the potential legal risks and uncertainties associated with its use. Any use beyond personal or research purposes, including commercial applications, may carry legal risks and you assume full responsibility for compliance with all applicable licenses and laws.
I recommend consulting with legal counsel to ensure your use of this model complies with all relevant licenses and regulations.
## Merge Details
### Merge Method
I found della_linear to be the most effective method for merging a Llama 3 model with Llama 3.1 out of a dozen or so different tests.
You can apply a higher density setting for sure. I went up to 0.5 density with an epsilon of 0.1 without any problems, and you could probably go higher than that, but I think this version with the lower density came out a little smarter and worked better for this particular pairing.
### Configuration
The following [mergekit](https://github.com/arcee-ai/mergekit) YAML will reproduce this model.
```yaml
merge_method: della_linear
base_model: meta-llama/Meta-Llama-3.1-70B-Instruct
models:
- model: sophosympatheia/New-Dawn-Llama-3-70B-32K-v1.0
parameters:
weight:
- filter: v_proj
value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
- filter: o_proj
value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
- filter: up_proj
value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
- filter: gate_proj
value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
- filter: down_proj
value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
- value: 0
density: 0.25
epsilon: 0.05
lambda: 1.0
- model: meta-llama/Meta-Llama-3.1-70B-Instruct
parameters:
weight: 1.0
density:
- filter: v_proj
value: [1, 1, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 1, 1]
- filter: o_proj
value: [1, 1, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 1, 1]
- filter: up_proj
value: [1, 1, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 1, 1]
- filter: gate_proj
value: [1, 1, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 1, 1]
- filter: down_proj
value: [1, 1, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 1, 1]
- value: 0.5
epsilon:
- filter: v_proj
value: [0, 0, 0.05, 0.05, 0.07, 0.1, 0.07, 0.05, 0.05, 0, 0]
- filter: o_proj
value: [0, 0, 0.05, 0.05, 0.07, 0.1, 0.07, 0.05, 0.05, 0, 0]
- filter: up_proj
value: [0, 0, 0.05, 0.05, 0.07, 0.1, 0.07, 0.05, 0.05, 0, 0]
- filter: gate_proj
value: [0, 0, 0.05, 0.05, 0.07, 0.1, 0.07, 0.05, 0.05, 0, 0]
- filter: down_proj
value: [0, 0, 0.05, 0.05, 0.07, 0.1, 0.07, 0.05, 0.05, 0, 0]
- value: 0.1
lambda: 1.0
dtype: float16
tokenizer_source: base
``` |