|
--- |
|
language: |
|
- fr |
|
- it |
|
- de |
|
- es |
|
- en |
|
license: apache-2.0 |
|
base_model: mistralai/Mixtral-8x7B-Instruct-v0.1 |
|
inference: |
|
parameters: |
|
temperature: 0.5 |
|
widget: |
|
- messages: |
|
- role: user |
|
content: What is your favorite condiment? |
|
extra_gated_description: If you want to learn more about how we process your personal |
|
data, please read our <a href="https://mistral.ai/terms/">Privacy Policy</a>. |
|
tags: |
|
- llama-cpp |
|
- gguf |
|
--- |
|
|
|
# Supa-AI/Mixtral-8x7B-Instruct-v0.1-gguf |
|
This model was converted to GGUF format from [`mistralai/Mixtral-8x7B-Instruct-v0.1`](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) using llama.cpp. |
|
Refer to the [original model card](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) for more details on the model. |
|
|
|
## Available Versions |
|
- `Mixtral-8x7B-Instruct-v0.1.q4_0.gguf` (q4_0) |
|
- `Mixtral-8x7B-Instruct-v0.1.q4_1.gguf` (q4_1) |
|
- `Mixtral-8x7B-Instruct-v0.1.q5_0.gguf` (q5_0) |
|
- `Mixtral-8x7B-Instruct-v0.1.q5_1.gguf` (q5_1) |
|
- `Mixtral-8x7B-Instruct-v0.1.q8_0.gguf` (q8_0) |
|
- `Mixtral-8x7B-Instruct-v0.1.q3_k_s.gguf` (q3_K_S) |
|
- `Mixtral-8x7B-Instruct-v0.1.q3_k_m.gguf` (q3_K_M) |
|
- `Mixtral-8x7B-Instruct-v0.1.q3_k_l.gguf` (q3_K_L) |
|
- `Mixtral-8x7B-Instruct-v0.1.q4_k_s.gguf` (q4_K_S) |
|
- `Mixtral-8x7B-Instruct-v0.1.q4_k_m.gguf` (q4_K_M) |
|
- `Mixtral-8x7B-Instruct-v0.1.q5_k_s.gguf` (q5_K_S) |
|
- `Mixtral-8x7B-Instruct-v0.1.q5_k_m.gguf` (q5_K_M) |
|
- `Mixtral-8x7B-Instruct-v0.1.q6_k.gguf` (q6_K) |
|
|
|
## Use with llama.cpp |
|
Replace `FILENAME` with one of the above filenames. |
|
|
|
### CLI: |
|
```bash |
|
llama-cli --hf-repo Supa-AI/Mixtral-8x7B-Instruct-v0.1-gguf --hf-file FILENAME -p "Your prompt here" |
|
``` |
|
|
|
### Server: |
|
```bash |
|
llama-server --hf-repo Supa-AI/Mixtral-8x7B-Instruct-v0.1-gguf --hf-file FILENAME -c 2048 |
|
``` |
|
|
|
## Model Details |
|
- **Original Model:** [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) |
|
- **Format:** GGUF |
|
|
