README.md

---
base_model: ArliAI/QwQ-32B-ArliAI-RpR-v3
language:
- en
license: apache-2.0
tags:
- llama-cpp
- gguf-my-repo
thumbnail: https://cdn-uploads.huggingface.co/production/uploads/6625f4a8a8d1362ebcc3851a/coilCTGeL0OUYr9PA9zna.jpeg
---

# Triangle104/QwQ-32B-ArliAI-RpR-v3-Q3_K_M-GGUF
This model was converted to GGUF format from [`ArliAI/QwQ-32B-ArliAI-RpR-v3`](https://huggingface.co/ArliAI/QwQ-32B-ArliAI-RpR-v3) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
Refer to the [original model card](https://huggingface.co/ArliAI/QwQ-32B-ArliAI-RpR-v3) for more details on the model.

---
RpR (RolePlay with Reasoning) is a new series of models from ArliAI. This series builds directly upon the successful dataset curation methodology and training methods developed for the RPMax series.

RpR models use the same curated, deduplicated RP and creative writing
 dataset used for RPMax, with a focus on variety to ensure high 
creativity and minimize cross-context repetition. Users familiar with 
RPMax will recognize the unique, non-repetitive writing style unlike 
other finetuned-for-RP models.

With the release of QwQ as the first high performing open-source 
reasoning model that can be easily trained, it was clear that the 
available instruct and creative writing reasoning datasets contains only
 one response per example. This is type of single response dataset used 
for training reasoning models causes degraded output quality in long 
multi-turn chats. Which is why Arli AI decided to create a real RP model
 capable of long multi-turn chat with reasoning.

In order to create RpR, we first had to actually create the reasoning
 RP dataset by re-processing our existing known-good RPMax dataset into a
 reasoning dataset. This was possible by using the base QwQ Instruct 
model itself to create the reasoning process for every turn in the RPMax
 dataset conversation examples, which is then further refined in order 
to make sure the reasoning is in-line with the actual response examples 
from the dataset.

Another important thing to get right is to make sure the model is 
trained on examples that present reasoning blocks in the same way as it 
encounters it during inference. Which is, never seeing the reasoning 
blocks in it's context. In order to do this, the training run was 
completed using axolotl with manual template-free segments dataset in 
order to make sure that the model is never trained to see the reasoning 
block in the context. Just like how the model will be used during 
inference time.

The result of training QwQ on this dataset with this method are 
consistently coherent and interesting outputs even in long multi-turn RP
 chats. This is as far as we know the first true correctly-trained 
reasoning model trained for RP and creative writing.

You can access the model at https://arliai.com and we also have a models ranking page at https://www.arliai.com/models-ranking

Ask questions in our new Discord Server https://discord.com/invite/t75KbPgwhk or on our subreddit https://www.reddit.com/r/ArliAI/

---
## Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)

```bash
brew install llama.cpp

```
Invoke the llama.cpp server or the CLI.

### CLI:
```bash
llama-cli --hf-repo Triangle104/QwQ-32B-ArliAI-RpR-v3-Q3_K_M-GGUF --hf-file qwq-32b-arliai-rpr-v3-q3_k_m.gguf -p "The meaning to life and the universe is"
```

### Server:
```bash
llama-server --hf-repo Triangle104/QwQ-32B-ArliAI-RpR-v3-Q3_K_M-GGUF --hf-file qwq-32b-arliai-rpr-v3-q3_k_m.gguf -c 2048
```

Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.
```
git clone https://github.com/ggerganov/llama.cpp
```

Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
```
cd llama.cpp && LLAMA_CURL=1 make
```

Step 3: Run inference through the main binary.
```
./llama-cli --hf-repo Triangle104/QwQ-32B-ArliAI-RpR-v3-Q3_K_M-GGUF --hf-file qwq-32b-arliai-rpr-v3-q3_k_m.gguf -p "The meaning to life and the universe is"
```
or 
```
./llama-server --hf-repo Triangle104/QwQ-32B-ArliAI-RpR-v3-Q3_K_M-GGUF --hf-file qwq-32b-arliai-rpr-v3-q3_k_m.gguf -c 2048
```