peft text-generation en license:apache-2.0 region:us
---
language:
- en
library_name: peft
license: apache-2.0
pipeline_tag: text-generation
inference: false
---

# PEFT Model Fine-tuned on UAE QA Pairs

This repository contains a fine-tuned model based on the PEFT framework for question answering tasks. The model has been trained on a dataset of question and answer pairs related to the UAE.

## Installation

Before using the model, make sure to install the necessary packages:

```sh
pip install transformers
pip install torch torchvision
pip install peft
```
## Usage
The model can be used for generating responses to prompts. Here is an example:

```python
from peft import PeftModel, PeftConfig
from transformers import  AutoModelForCausalLM, AutoTokenizer
import torch

peft_model_id = "insomeniaT/falcon-7b-uae-qapairs-67"

config = PeftConfig.from_pretrained(peft_model_id)
model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, trust_remote_code=True)
model = PeftModel.from_pretrained(model, peft_model_id)

tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-7b")
tokenizer.pad_token = tokenizer.eos_token

text = "### Human: What is the minimum requirement for the UAE's GCC residency?? ### Assistant: "
device = "cuda:0"

inputs = tokenizer(text, return_tensors="pt")
inputs.to(device)
model.to(device)

outputs = model.generate(input_ids=inputs["input_ids"], attention_mask=inputs["attention_mask"], max_new_tokens=300, pad_token_id=tokenizer.eos_token_id)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)
```
