gguf endpoints_compatible region:us
---
{}
---
Quantization made by Richard Erkhov.

[Github](https://github.com/RichardErkhov)

[Discord](https://discord.gg/pvy7H8DZMG)

[Request more models](https://github.com/RichardErkhov/quant_request)


RAGPT-2_unfunctional - GGUF
- Model creator: https://huggingface.co/BueormLLC/
- Original model: https://huggingface.co/BueormLLC/RAGPT-2_unfunctional/


| Name | Quant method | Size |
| ---- | ---- | ---- |
| [RAGPT-2_unfunctional.Q2_K.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q2_K.gguf) | Q2_K | 0.08GB |
| [RAGPT-2_unfunctional.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.IQ3_XS.gguf) | IQ3_XS | 0.08GB |
| [RAGPT-2_unfunctional.IQ3_S.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.IQ3_S.gguf) | IQ3_S | 0.08GB |
| [RAGPT-2_unfunctional.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q3_K_S.gguf) | Q3_K_S | 0.08GB |
| [RAGPT-2_unfunctional.IQ3_M.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.IQ3_M.gguf) | IQ3_M | 0.09GB |
| [RAGPT-2_unfunctional.Q3_K.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q3_K.gguf) | Q3_K | 0.09GB |
| [RAGPT-2_unfunctional.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q3_K_M.gguf) | Q3_K_M | 0.09GB |
| [RAGPT-2_unfunctional.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q3_K_L.gguf) | Q3_K_L | 0.1GB |
| [RAGPT-2_unfunctional.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.IQ4_XS.gguf) | IQ4_XS | 0.1GB |
| [RAGPT-2_unfunctional.Q4_0.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q4_0.gguf) | Q4_0 | 0.1GB |
| [RAGPT-2_unfunctional.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.IQ4_NL.gguf) | IQ4_NL | 0.1GB |
| [RAGPT-2_unfunctional.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q4_K_S.gguf) | Q4_K_S | 0.1GB |
| [RAGPT-2_unfunctional.Q4_K.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q4_K.gguf) | Q4_K | 0.11GB |
| [RAGPT-2_unfunctional.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q4_K_M.gguf) | Q4_K_M | 0.11GB |
| [RAGPT-2_unfunctional.Q4_1.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q4_1.gguf) | Q4_1 | 0.11GB |
| [RAGPT-2_unfunctional.Q5_0.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q5_0.gguf) | Q5_0 | 0.11GB |
| [RAGPT-2_unfunctional.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q5_K_S.gguf) | Q5_K_S | 0.11GB |
| [RAGPT-2_unfunctional.Q5_K.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q5_K.gguf) | Q5_K | 0.12GB |
| [RAGPT-2_unfunctional.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q5_K_M.gguf) | Q5_K_M | 0.12GB |
| [RAGPT-2_unfunctional.Q5_1.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q5_1.gguf) | Q5_1 | 0.12GB |
| [RAGPT-2_unfunctional.Q6_K.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q6_K.gguf) | Q6_K | 0.13GB |
| [RAGPT-2_unfunctional.Q8_0.gguf](https://huggingface.co/RichardErkhov/BueormLLC_-_RAGPT-2_unfunctional-gguf/blob/main/RAGPT-2_unfunctional.Q8_0.gguf) | Q8_0 | 0.17GB |




Original model description:
---
license: mit
datasets:
- neural-bridge/rag-dataset-12000
- neural-bridge/rag-dataset-1200
language:
- en
---
# VERY IMPORTANT
- This model is in alpha phase and is NOT yet recommended for use.
- This model is obsolete today, recommended model [here.](https://huggingface.co/BueormLLC/RAGPT-2_Turbo)
# RAGPT-2 (unfunctional): Fine-tuned GPT-2 for Context-Based Question Answering

## Model Description

RAGPT-2 is a fine-tuned version of [GPT-2 small](https://huggingface.co/BueormLLC/CleanGPT), specifically adapted for context-based question answering tasks. This model has been trained to generate relevant answers based on a given context and question, similar to a Retrieval-Augmented Generation (RAG) system.

### Key Features

- Based on the GPT-2 small architecture (124M parameters)
- Fine-tuned on the "neural-bridge/rag-dataset-12000" and others dataset from Hugging Face
- Capable of generating answers based on provided context and questions
- Suitable for various question-answering applications

## Training Data

The model was fine-tuned using the "neural-bridge/rag-dataset-12000" and "neural-bridge/rag-dataset-1200" dataset, which contains:
- Context passages
- Questions related to the context
- Corresponding answers

## Fine-tuning Process

The fine-tuning process involved:
1. Loading the pre-trained GPT-2 small model
2. Preprocessing the dataset to combine context, question, and answer into a single text
3. Training the model to predict the next token given the context and question

### Hyperparameters

- Base model: GPT-2 small
- Number of training epochs: 8
- Batch size: 4
- Learning rate: Default AdamW optimizer settings
- Max sequence length: 512 tokens

## Usage

To use the model:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("BueormLLC/RAGPT-2_unfunctional")
model = AutoModelForCausalLM.from_pretrained("BueormLLC/RAGPT-2_unfunctional")

context = "Mount Everest is the highest mountain in the world, with a height of 8,848 meters."
question = "What is the height of Mount Everest?"
input_text = f"Context: {context}\nquestion: {question}\nanswer:"

input_ids = tokenizer.encode(input_text, return_tensors="pt")
output = model.generate(input_ids, max_length=150, num_return_sequences=1)
answer = tokenizer.decode(output[0], skip_special_tokens=True)

print(f"Respuesta generada: {answer}")
```

## Limitations

- The model's knowledge is limited to its training data and the base GPT-2 model.
- It may sometimes generate irrelevant or incorrect answers, especially for topics outside its training domain.
- The model does not have access to external information or real-time data.

## Ethical Considerations

Users should be aware that this model, like all language models, may reflect biases present in its training data. It should not be used as a sole source of information for critical decisions.

## Future Improvements

- Fine-tuning on a larger and more diverse dataset
- Experimenting with larger base models (e.g., GPT-2 medium or large)
- Implementing techniques to improve factual accuracy and reduce hallucinations

## Support us

- [Paypal](https://paypal.me/bueorm)
- [Patreon](https://patreon.com/bueorm)
### We appreciate your support, without you we could not do what we do.

## Citation

If you use this model in your research, please cite:

```
@misc{RAGPT,
  author = {Bueorm},
  title = {RAGPT-2: Fine-tuned GPT-2 for Context-Based Question Answering},
  year = {2024},
  publisher = {GitHub},
  journal = {None},
  howpublished = {\url{https://huggingface.co/BueormLLC/RAGPT-2_unfunctional}}
}
```

