marcelone
/

Llama-3.1-Swallow-8B-Instruct-v0.5-gguf

Model card Files Files and versions

marcelone commited on Jul 27

Commit

08a0944

·

verified ·

1 Parent(s): a974098

Update README.md

Files changed (1) hide show

README.md +6 -0

README.md CHANGED Viewed

@@ -7,6 +7,12 @@ base_model:
 - tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.5
 base_model_relation: quantized
 ---
 # Swallow-8B-it-v05-gguf-q8_0-mixed-v1
 * **Quantization Type**: Mixed Precision (`bf16`, `q4_K`, `q5_K`, `q6_K`, `q8_0`)
 * **Bits Per Weight (BPW)**: `8.01`

 - tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.5
 base_model_relation: quantized
 ---
+# Swallow-8B-it-v05-gguf-q6_k-mixed-v1
+* **Quantization Type**: Mixed Precision (`q5_K`, `q6_K`, `q8_0`)
+* **Bits Per Weight (BPW)**: `7.13`
+# Swallow-8B-it-v05-gguf-q6_k-mixed-v2
+* **Quantization Type**: Mixed Precision (`q6_K`, `q8_0`)
+* **Bits Per Weight (BPW)**: `7.50`
 # Swallow-8B-it-v05-gguf-q8_0-mixed-v1
 * **Quantization Type**: Mixed Precision (`bf16`, `q4_K`, `q5_K`, `q6_K`, `q8_0`)
 * **Bits Per Weight (BPW)**: `8.01`