Upload 15 files

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,190 @@
+---
+base_model:
+- trashpanda-org/QwQ-32B-Snowdrop-v0
+base_model_relation: quantized
+library_name: transformers
+tags:
+- mergekit
+- mergekitty
+- merge
+---
+## Quantized using the default exllamav3 (0.0.1) quantization process.
+
+- Original model: https://huggingface.co/trashpanda-org/QwQ-32B-Snowdrop-v0
+- exllamav3: https://github.com/turboderp-org/exllamav3
+---
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/Tdn0PJBFnG3J6UcjO9G94.png)
+
+<sup>QwQwQwQwQwQ and Marigold met at a party and hit it off...</sup>
+<p><b>Has's notes</b>: it's actually pretty damn good?!</p>
+
+<p><b>Severian's notes</b>: R1 at home for RP, literally. Able to handle my cards with gimmicks and subtle tricks in them. With a good reasoning starter+prompt, I'm getting consistently-structured responses that have a good amount of variation across them still while rerolling. Char/scenario portrayal is good despite my focus on writing style, lorebooks are properly referenced at times. Slop doesn't seem to be too much of an issue with thinking enabled. Some user impersonation is rarely observed. Prose is refreshing if you take advantage of what I did (writing style fixation). I know I said Marigold would be my daily driver, but this one is that now, it's that good.</p>
+
+## Recommended settings
+
+<p><b>Context/instruct template</b>: ChatML. <s>Was definitely not tested with ChatML instruct and Mistral v7 template, nuh-uh.</s></p>
+
+<p><b>Samplers</b>: temperature at 0.9, min_p at 0.05, top_a at 0.3, TFS at 0.75, repetition_penalty at 1.03, DRY if you have access to it.</p>
+
+A virt-io derivative prompt worked best during our testing, but feel free to use what you like.
+
+Master import for ST: [https://files.catbox.moe/b6nwbc.json](https://files.catbox.moe/b6nwbc.json)
+
+## Reasoning
+
+Feel free to test whichever reasoning setup you're most comfortable with, but here's a recommendation from me. My prompt has a line that says:
+
+```
+Style Preference: Encourage the usage of a Japanese light novel writing style.
+```
+
+Deciding to fixate on that, my reasoning starter is:
+
+```
+<think>Okay, in this scenario, before responding I need to consider the writing style referenced in the prompt, which is
+```
+
+What this did for me, at least during testing is that it gave the reasoning a structure to follow across rerolls, seeking out that part of the prompt consistently.
+See below:
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/Mw6h-mmZ0TcQrtNPGdCsV.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/JeSEYwTJofYRqLHKm8glm.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/nDSAJK7HDc-bKFXe6ER_q.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/j2slz1hEKsEM3bVrR--xs.png)
+
+But the responses were still varied, because the next few paragraphs after these delved into character details, so on and so forth. Might want to experiment and make your own thinking/reasoning starter that focuses on what you hope to get out of the responses for best results.
+
+— Severian
+
+## Thank you!
+
+Big thanks to the folks in the trashpanda-org discord for testing and sending over some logs!
+
+## Reviews
+
+> PROS:
+> 
+> In 10 swipes, had only two minor instances of speaking for {{user}}. (Can probably be fixed with a good prompt, though.)
+> 
+> Creativity: 8/10 swipes provided unique text for 90% of the response, almost no cliché phrases.
+>
+> Takes personality of characters into account, sticking to it well. Even without a lorebook to support it was able to retain lore-specific terms and actually remember which meant which.
+>
+> NPCs: In 6/10 swipes NPC characters also partook in action, sticking to bits of information provided about them in opening message. Some of them even had their unique speech patterns. (Certain with a proper lorebook it would cook.)
+>
+> Unfiltered, graphic descriptions of fight scenes. Magic, physical attacks - everything was taken into account with no holding back.
+>
+> CONS:
+>
+> Some swipes were a bit OOC. Some swipes were bland, providing little to no input or any weight on the roleplay context.
+>
+> Out of all models I've tried recently, this one definitely has most potential. With proper prompting I think this beast would be genuinely one of the best models for unique scenarios.
+
+— Sellvene
+
+> It's one of the -maybe THE- best small thinking models right now. It sticks to character really well, slops are almost non-existent though they are still there of course, it proceeds with the story well and listens to the prompt. I LOVE R1 but I love snowdrop even more right now because answers feel more geniune and less agressive compared to R1.
+
+— Carmenta
+
+> Writes better than GPT 4.5. Overall, I think censorship is fucking up more unhinged bots and it's too tame for my liking. Another thing I noticed is that, it's sticking too much to being "right" to the character and too afraid to go off the rails.
+
+— Myscell
+
+> I'm fainting, the character breakdown in it's thinking is similar like R1 does. Character handling looks amazing. Broo if a merge this good then, I'm looking forward to that QwQ finetune.
+
+— Sam
+
+> Negligible slop, no positivity bias which is good though. I like the model so far, R1 at home.
+
+— Raihanbook
+
+> Overall, I think this is a real solid model. Cot is great, listens to my prompt extremely well. Number 1 for reasoning, honestly. And the way it portrays the character and persona details? Perfect. Narration, perfect. I have very little complaints about this model, ya'll cooked.
+
+— Moothdragon
+
+> On my end, posivity bias isn't really there 🤔 Character and scenario portrayal is good. The prose too, I like it. Between this and Marigold, I feel like I can lean into snowboard (I mean Snowdrop) more. For now though, it is still Marigold.
+
+— Azula
+
+> Honestly i am impressed and I like it.
+
+— OMGWTFBBQ
+
+> It's pretty damn good. Better than Mullein, I think.
+
+— br
+
+> So far, it fucking SLAPS. I don't think it's tried to pull POV once yet.
+
+— Overloke
+
+## Just us having fun, don't mind it
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/ci-okzqZjNgk-CgYFVCNO.png)
+
+## Some logs
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/jMlZatm5wvN8KNRlWeNbK.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/o4PQb0FbPrU_PoIT5pzDu.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/jFTpiw65LHkel3EzR6IRV.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/vwdKSjICLqua98TXDsv-W.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/R7B7QsMjIdgzPT1-i4UI6.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/Xn2FPUPESWPjzOQQ4sdwp.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/tok2RfOE0BQHHkPOpVje3.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/vtkwpEEubUyZ3mYW6asru.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/4xWnSieopicxQtwjaO4Ri.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/dLlvlP4U_cWpd84e5Tqtd.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/8O7p_z6EN9Tf7Rr52RXm7.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/m_6B_uJAdaGlq8UlYIPIm.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/daEqb2Qi0pA6UobNsJota.png)
+(After a session started with Gemini)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/DQMan-Ywm-meyFtJGVWlF.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/XKEp3p7UZ8mlBeOxAiXEa.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/wSA-eBIAh9Ru0BE5HeIJ-.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/qd1XFGtopRQxQqHrs98wh.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/AzGgKBtTBzwUSx0So4_pS.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/VyEZvFY6yL4OkTBioqdaP.png)
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/675a77cf99ca23af9daacccc/cTgYWUmnLORfX1fZZw-1f.png)
+
+## Merge Details
+### Merge Method
+
+This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [Qwen/Qwen2.5-32B](https://huggingface.co/Qwen/Qwen2.5-32B) as a base.
+
+### Models Merged
+
+The following models were included in the merge:
+* [trashpanda-org/Qwen2.5-32B-Marigold-v0](https://huggingface.co/trashpanda-org/Qwen2.5-32B-Marigold-v0)
+* [Qwen/QwQ-32B](https://huggingface.co/Qwen/QwQ-32B)
+* [trashpanda-org/Qwen2.5-32B-Marigold-v0-exp](https://huggingface.co/trashpanda-org/Qwen2.5-32B-Marigold-v0-exp)
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+models:
+  - model: trashpanda-org/Qwen2.5-32B-Marigold-v0-exp
+    parameters:
+      weight: 1
+      density: 1
+  - model: trashpanda-org/Qwen2.5-32B-Marigold-v0
+    parameters:
+      weight: 1
+      density: 1
+  - model: Qwen/QwQ-32B
+    parameters:
+      weight: 0.9
+      density: 0.9
+merge_method: ties
+base_model: Qwen/Qwen2.5-32B
+parameters:
+  weight: 0.9
+  density: 0.9
+  normalize: true
+  int8_mask: true
+tokenizer_source: Qwen/Qwen2.5-32B-Instruct
+dtype: bfloat16
+
+
+```

added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

config.json

@@ -0,0 +1,40 @@
+{
+    "_name_or_path": "Qwen/Qwen2.5-32B",
+    "architectures": [
+        "Qwen2ForCausalLM"
+    ],
+    "attention_dropout": 0.0,
+    "bos_token_id": 151643,
+    "eos_token_id": 151643,
+    "hidden_act": "silu",
+    "hidden_size": 5120,
+    "initializer_range": 0.02,
+    "intermediate_size": 27648,
+    "max_position_embeddings": 131072,
+    "max_window_layers": 64,
+    "model_type": "qwen2",
+    "num_attention_heads": 40,
+    "num_hidden_layers": 64,
+    "num_key_value_heads": 8,
+    "rms_norm_eps": 1e-05,
+    "rope_scaling": null,
+    "rope_theta": 1000000.0,
+    "sliding_window": 131072,
+    "tie_word_embeddings": false,
+    "torch_dtype": "bfloat16",
+    "transformers_version": "4.49.0",
+    "use_cache": true,
+    "use_sliding_window": false,
+    "vocab_size": 151665,
+    "quantization_config": {
+        "quant_method": "exl3",
+        "version": "0.0.1",
+        "bits": 4.0,
+        "head_bits": 6,
+        "calibration": {
+            "rows": 100,
+            "cols": 2048
+        },
+        "out_scales": "auto"
+    }
+}

mergekitty_config.yml

@@ -0,0 +1,23 @@
+models:
+  - model: trashpanda-org/Qwen2.5-32B-Marigold-v0-exp
+    parameters:
+      weight: 1
+      density: 1
+  - model: trashpanda-org/Qwen2.5-32B-Marigold-v0
+    parameters:
+      weight: 1
+      density: 1
+  - model: Qwen/QwQ-32B
+    parameters:
+      weight: 0.9
+      density: 0.9
+merge_method: ties
+base_model: Qwen/Qwen2.5-32B
+parameters:
+  weight: 0.9
+  density: 0.9
+  normalize: true
+  int8_mask: true
+tokenizer_source: Qwen/Qwen2.5-32B-Instruct
+dtype: bfloat16
+

model-00001-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f86df203f51ba095cc4f6e64eb530457ae096cba38dddb328c01b629b41a04d6
+size 8387675040

model-00002-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:348084430b03fa647c6235e4d0ddbaf4b920f82ea2b7c58c8623a8f4b3b85706
+size 8543277872

model-00003-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0aa5e95c129dc860b36daf8b4c4f3996fa6c522e00de24205c131b1c5aabd4d1
+size 826868976

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c0382117ea329cdf097041132f6d735924b697924d6f6fc3945713e96ce87539
+size 7031645

tokenizer_config.json

@@ -0,0 +1,208 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}