|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- FreedomIntelligence/RAG-Instruct |
|
language: |
|
- en |
|
metrics: |
|
- accuracy |
|
base_model: |
|
- meta-llama/Llama-3.1-8B |
|
pipeline_tag: text-generation |
|
--- |
|
## Introduction |
|
|
|
RAG-Instruct is a method for generating diverse and high-quality RAG instruction data. It synthesizes instruction datasets based on any source corpus, leveraging the following approaches: |
|
|
|
- **Five RAG paradigms**, which represent diverse query-document relationships to enhance model generalization across tasks. |
|
- **Instruction simulation**, which enriches instruction diversity and quality by utilizing the strengths of existing instruction datasets. |
|
|
|
Using this approach, we constructed [RAG-Instruct](https://huggingface.co/datasets/FreedomIntelligence/RAG-Instruct), covering a wide range of RAG scenarios and tasks. |
|
|
|
Our RAG-Instruct-Llama3-8B is trained on [RAG-Instruct](https://huggingface.co/datasets/FreedomIntelligence/RAG-Instruct) data, which significantly enhances the RAG ability of LLMs, demonstrating remarkable improvements in RAG performance across various tasks. |
|
|
|
| Model | WQA (acc) | PQA (acc) | TQA (acc) | OBQA (EM) | Pub (EM) | ARC (EM) | 2WIKI (acc) | HotP (acc) | MSQ (acc) | CFQA (EM) | PubMed (EM) | |
|
|--------------------------------|-----------|-----------|-----------|-----------|----------|----------|-------------|------------|-----------|-----------|-------------| |
|
| Llama3.1-8B | 59.5 | 60.8 | 73.4 | 82.0 | 56.7 | 77.1 | 65.6 | 45.6 | 18.7 | 56.5 | 73.9 | |
|
| Llama3.1-8B + **RAG-Instruct** | 69.7 | 68.4 | 79.3 | 84.8 | 77.2 | 79.9 | 79.3 | 56.4 | 30.3 | 57.8 | 77.0 | |
|
|
|
|
|
# <span>Usage</span> |
|
RAG-Instruct-Llama3-8B can be used just like `Llama-3.1-8B-Instruct`. You can deploy it with tools like [vllm](https://github.com/vllm-project/vllm) or [Sglang](https://github.com/sgl-project/sglang), or perform direct inference: |
|
```python |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
# Load the model and tokenizer |
|
model = AutoModelForCausalLM.from_pretrained("FreedomIntelligence/RAG-Instruct-Llama3-8B",torch_dtype="auto",device_map="auto") |
|
tokenizer = AutoTokenizer.from_pretrained("FreedomIntelligence/RAG-Instruct-Llama3-8B") |
|
|
|
# Example input |
|
input_text = """### Paragraph: |
|
[1] structure is at risk from new development... |
|
[2] as Customs and Excise stores... |
|
[3] Powis Street is partly underway... |
|
... |
|
|
|
### Instruction: |
|
Which organization is currently using a building in Woolwich that holds historical importance? |
|
""" |
|
|
|
# Tokenize and prepare input |
|
messages = [{"role": "user", "content": input_text}] |
|
inputs = tokenizer(tokenizer.apply_chat_template(messages, tokenize=False,add_generation_prompt=True), return_tensors="pt").to(model.device) |
|
|
|
# Generate output |
|
outputs = model.generate(**inputs, max_new_tokens=2048) |
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
``` |
|
|
|
## Citation |
|
``` |
|
@misc{liu2024raginstructboostingllmsdiverse, |
|
title={RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructions}, |
|
author={Wanlong Liu and Junying Chen and Ke Ji and Li Zhou and Wenyu Chen and Benyou Wang}, |
|
year={2024}, |
|
eprint={2501.00353}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL}, |
|
url={https://arxiv.org/abs/2501.00353}, |
|
} |
|
``` |
