File size: 4,631 Bytes
54ad099 8d24d4f 54ad099 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
---
license: apache-2.0
language:
- pl
base_model:
- CYFRAGOVPL/PLLuM-8x7B-chat
---
# PLLuM-8x7B-chat GGUF Quantizations by Nondzu
DISCLAIMER: This is a quantized version of an existing model [PLLuM-8x7B-chat](https://huggingface.co/CYFRAGOVPL/PLLuM-8x7B-chat). I am not the author of the original model. I am only hosting the quantized models. I do not take any responsibility for the models.
## Prompt Format
Use the following prompt structure:
```
???
```
## Available Files
Below is a list of available quantized model files along with their quantization type, file size, and a short description.
| Filename | Quant Type | File Size | Description |
| ------------------------------------------------------------------------------------- | ---------- | --------- | --------------------------------------------------------------------------------------------- |
| [PLLuM-8x7B-chat-Q2_K.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main) | Q2_K | 17 GB | Very low quality but surprisingly usable. |
| [PLLuM-8x7B-chat-Q3_K.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main) | Q3_K | 21 GB | Low quality, suitable for setups with very limited RAM. |
| [PLLuM-8x7B-chat-Q3_K_L.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main) | Q3_K_L | 23 GB | High quality; recommended for quality-focused usage. |
| [PLLuM-8x7B-chat-Q3_K_M.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main) | Q3_K_M | 21 GB | Very high quality, near perfect output – recommended. |
| [PLLuM-8x7B-chat-Q3_K_S.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main) | Q3_K_S | 20 GB | Moderate quality with improved space efficiency. |
| [PLLuM-8x7B-chat-Q4_K.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main) | Q4_K | 27 GB | Good quality for standard use. |
| [PLLuM-8x7B-chat-Q4_K_M.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main) | Q4_K_M | 27 GB | Default quality for most use cases – recommended. |
| [PLLuM-8x7B-chat-Q4_K_S.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main) | Q4_K_S | 25 GB | Slightly lower quality with enhanced space savings – recommended when size is a priority. |
| [PLLuM-8x7B-chat-Q5_0.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main) | Q5_0 | 31 GB | Extremely high quality – the maximum quant available. |
| [PLLuM-8x7B-chat-Q5_K.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main) | Q5_K | 31 GB | Very high quality – recommended for demanding use cases. |
| [PLLuM-8x7B-chat-Q5_K_M.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main) | Q5_K_M | 31 GB | High quality – recommended. |
| [PLLuM-8x7B-chat-Q5_K_S.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main) | Q5_K_S | 31 GB | High quality, offered as an alternative with minimal quality loss. |
| [PLLuM-8x7B-chat-Q6_K.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main) | Q6_K | 36 GB | Very high quality with quantized embed/output weights. |
| [PLLuM-8x7B-chat-Q8_0.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main) | Q8_0 | 47 GB | Maximum quality quantization. |
## Downloading Using Hugging Face CLI
<details>
<summary>Click to view download instructions</summary>
First, ensure you have the Hugging Face CLI installed:
```bash
pip install -U "huggingface_hub[cli]"
```
Then, target a specific file to download:
```bash
huggingface-cli download Nondzu/PLLuM-8x7B-chat-GGUF --include "PLLuM-8x7B-chat-Q4_K_M.gguf" --local-dir ./
```
For larger files, you can specify a new local directory (e.g., `PLLuM-8x7B-chat-Q8_0`) or download them directly into the current directory (`./`).
</details> |