Triangle104/Dans-SakuraKaze-V1.0.0-12b-Q5_K_M-GGUF

This model was converted to GGUF format from PocketDoc/Dans-SakuraKaze-V1.0.0-12b using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

---

A model based on Dans-PersonalityEngine-V1.1.0-12b with a focus on character RP, visual novel style group chats, old school text adventures, and co-writing.

Key Details

BASE MODEL: PocketDoc/Dans-PersonalityEngine-V1.1.0-12b LICENSE: apache-2.0 LANGUAGE: English CONTEXT LENGTH: 32768 tokens

Sponsored by Chub.AI

Recommended Settings

TEMPERATURE: 1.0 TOP_P: 0.95 MIN_P: 0.05

Prompting Format

The model uses standard "ChatML" format:

<|im_start|>system system prompt<|im_end|> <|im_start|>user Hi there!<|im_end|> <|im_start|>assistant Nice to meet you!<|im_end|>

SillyTavern Templates

Context Template

{ "story_string": "<|im_start|>system\n{{#if system}}{{system}}\n{{/if}}{{#if wiBefore}}{{wiBefore}}\n{{/if}}{{#if description}}{{description}}\n{{/if}}{{#if personality}}{{char}}'s personality: {{personality}}\n{{/if}}{{#if scenario}}Scenario: {{scenario}}\n{{/if}}{{#if wiAfter}}{{wiAfter}}\n{{/if}}{{#if persona}}{{persona}}\n{{/if}}{{trim}}<|im_end|>\n", "example_separator": "", "chat_start": "", "use_stop_strings": false, "allow_jailbreak": false, "always_force_name2": false, "trim_sentences": false, "include_newline": false, "single_line": false, "name": "Dan-ChatML" }

Instruct Template

{ "system_prompt": "Write {{char}}'s actions and dialogue, user will write {{user}}'s.", "input_sequence": "<|im_start|>user\n", "output_sequence": "<|im_start|>assistant\n", "first_output_sequence": "", "last_output_sequence": "", "system_sequence_prefix": "", "system_sequence_suffix": "", "stop_sequence": "<|im_end|>", "wrap": false, "macro": true, "names": false, "names_force_groups": false, "activation_regex": "", "skip_examples": false, "output_suffix": "<|im_end|>\n", "input_suffix": "<|im_end|>\n", "system_sequence": "<|im_start|>system\n", "system_suffix": "<|im_end|>\n", "user_alignment_message": "", "last_system_sequence": "", "system_same_as_user": false, "first_input_sequence": "", "last_input_sequence": "", "name": "Dan-ChatML" }

A Chub.AI Sponsored Model

Sponsored by Chub.AI

Character Hub supported this model with 45 hours on a 2x A100 80GB system. This is only some of what they've provided me for training and I am very grateful for their contributions.

Character Hub has been supporting model development for quite a while now and they may be interested in your projects! Contact them through this google form.

Support Development

Development is limited by funding and resources. To help support:

• Contact on HF

• Email: [email protected]

---

Use with llama.cpp

Install llama.cpp through brew (works on Mac and Linux)

brew install llama.cpp

Invoke the llama.cpp server or the CLI.

CLI:

llama-cli --hf-repo Triangle104/Dans-SakuraKaze-V1.0.0-12b-Q5_K_M-GGUF --hf-file dans-sakurakaze-v1.0.0-12b-q5_k_m.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo Triangle104/Dans-SakuraKaze-V1.0.0-12b-Q5_K_M-GGUF --hf-file dans-sakurakaze-v1.0.0-12b-q5_k_m.gguf -c 2048

Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference through the main binary.

./llama-cli --hf-repo Triangle104/Dans-SakuraKaze-V1.0.0-12b-Q5_K_M-GGUF --hf-file dans-sakurakaze-v1.0.0-12b-q5_k_m.gguf -p "The meaning to life and the universe is"

or

./llama-server --hf-repo Triangle104/Dans-SakuraKaze-V1.0.0-12b-Q5_K_M-GGUF --hf-file dans-sakurakaze-v1.0.0-12b-q5_k_m.gguf -c 2048