gguf endpoints_compatible region:us conversational
---
{}
---
Quantization made by Richard Erkhov.

[Github](https://github.com/RichardErkhov)

[Discord](https://discord.gg/pvy7H8DZMG)

[Request more models](https://github.com/RichardErkhov/quant_request)


OrpoLlama-3.2-1B-V1 - GGUF
- Model creator: https://huggingface.co/bhuvana-ak7/
- Original model: https://huggingface.co/bhuvana-ak7/OrpoLlama-3.2-1B-V1/


| Name | Quant method | Size |
| ---- | ---- | ---- |
| [OrpoLlama-3.2-1B-V1.Q2_K.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q2_K.gguf) | Q2_K | 0.54GB |
| [OrpoLlama-3.2-1B-V1.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q3_K_S.gguf) | Q3_K_S | 0.6GB |
| [OrpoLlama-3.2-1B-V1.Q3_K.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q3_K.gguf) | Q3_K | 0.64GB |
| [OrpoLlama-3.2-1B-V1.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q3_K_M.gguf) | Q3_K_M | 0.64GB |
| [OrpoLlama-3.2-1B-V1.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q3_K_L.gguf) | Q3_K_L | 0.68GB |
| [OrpoLlama-3.2-1B-V1.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.IQ4_XS.gguf) | IQ4_XS | 0.7GB |
| [OrpoLlama-3.2-1B-V1.Q4_0.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q4_0.gguf) | Q4_0 | 0.72GB |
| [OrpoLlama-3.2-1B-V1.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.IQ4_NL.gguf) | IQ4_NL | 0.72GB |
| [OrpoLlama-3.2-1B-V1.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q4_K_S.gguf) | Q4_K_S | 0.72GB |
| [OrpoLlama-3.2-1B-V1.Q4_K.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q4_K.gguf) | Q4_K | 0.75GB |
| [OrpoLlama-3.2-1B-V1.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q4_K_M.gguf) | Q4_K_M | 0.75GB |
| [OrpoLlama-3.2-1B-V1.Q4_1.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q4_1.gguf) | Q4_1 | 0.77GB |
| [OrpoLlama-3.2-1B-V1.Q5_0.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q5_0.gguf) | Q5_0 | 0.83GB |
| [OrpoLlama-3.2-1B-V1.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q5_K_S.gguf) | Q5_K_S | 0.83GB |
| [OrpoLlama-3.2-1B-V1.Q5_K.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q5_K.gguf) | Q5_K | 0.85GB |
| [OrpoLlama-3.2-1B-V1.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q5_K_M.gguf) | Q5_K_M | 0.85GB |
| [OrpoLlama-3.2-1B-V1.Q5_1.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q5_1.gguf) | Q5_1 | 0.89GB |
| [OrpoLlama-3.2-1B-V1.Q6_K.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q6_K.gguf) | Q6_K | 0.95GB |
| [OrpoLlama-3.2-1B-V1.Q8_0.gguf](https://huggingface.co/RichardErkhov/bhuvana-ak7_-_OrpoLlama-3.2-1B-V1-gguf/blob/main/OrpoLlama-3.2-1B-V1.Q8_0.gguf) | Q8_0 | 1.23GB |




Original model description:
---
library_name: transformers
tags: []
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->
This model is a fine-tuned version of meta-llama/Llama-3.2-1B, using ORPO (Optimized Regularization for Prompt Optimization) Trainer. 
This model is fine-tuned using the mlabonne/orpo-dpo-mix-40k dataset.
Only 1000 data samples were used to train quickly using ORPO.


## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

The base model meta-llama/Llama-3.2-1B has been fine-tuned using ORPO on a few samples of mlabonne/orpo-dpo-mix-40k dataset.
The Llama 3.2 instruction-tuned text-only model is optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks. 
This fine-tuned version is aimed at improving the understanding of the context in prompts and thereby increasing  the interpretability of the model.


- **Finetuned from model [meta-llama/Llama-3.2-1B]**
- **Model Size: 1 Billion parameters**
- **Fine-tuning Method: ORPO**
- **Dataset: mlabonne/orpo-dpo-mix-40k**

## Evaluation

The model was evaluated on the following benchmarks, with the following performance metrics:
|  Tasks  |Version|Filter|n-shot| Metric |   |Value |   |Stderr|
|---------|------:|------|-----:|--------|---|-----:|---|-----:|
|hellaswag|      1|none  |     0|acc     |↑  |0.4772|±  |0.0050|
|         |       |none  |     0|acc_norm|↑  |0.6366|±  |0.0048|
|tinyMMLU|      0|none  |     0|acc_norm|↑  |0.4306|±  |   N/A|
|eq_bench|    2.1|none  |     0|eqbench          |↑  |-12.9709|±  |2.9658|
|        |       |none  |     0|percent_parseable|↑  | 92.9825|±  |1.9592|


