Rustamshry/Llama-AzerbaijaniGovQA

Model Card for Model ID

Model Details

This model is a fine-tuned version of Meta's LLaMA 3.2B Instruct (Llama-3.2-3B-Instruct) designed for Azerbaijani Question Answering tasks, particularly focused on government and public service-related queries. It was trained using a curated Azerbaijani government QA dataset to improve performance on civic information tasks.

Model Description

• Language(s) (NLP): Azerbaijani
• License: Llama 3.2
• Finetuned from model: unsloth/Llama-3.2-3B-Instruct
• Dataset: arzumanabbasov/azerbaijani-gov-qa

Uses

Direct Use

This model is useful for:

• Azerbaijani-language civic Q&A systems
• Chatbots for government portals
• Legal or bureaucratic FAQs in Azerbaijani

Bias, Risks, and Limitations

• May hallucinate answers if queries are ambiguous or out of domain.
• Trained on a small dataset; may not generalize to all government topics.
• No vision or multimodal input support.

How to Get Started with the Model

Use the code below to get started with the model.

from huggingface_hub import login
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

login(token="")  

tokenizer = AutoTokenizer.from_pretrained("unsloth/Llama-3.2-3B-Instruct",)
base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/Llama-3.2-3B-Instruct",
    device_map={"": 0}, token=""
)

model = PeftModel.from_pretrained(base_model,"Rustamshry/Llama-AzerbaijaniGovQA")

question = 
"""
Salam 2020 ci ilde aile qurmudam karantin vaxti mene İkili soyada kecmek haqiqinda melumat
vermemisdiler men tezelikce melumatim olub ikili sayad haqqinda men deyisdire bilerem soyadimi? ikili etmək istəyirem
"""

instruction = "Siz Azərbaycanda dövlət xidmətləri üzrə ixtisaslaşmış süni intellekt üzrə köməkçisiniz."

prompt = (
    f"### Talimat:\n{instruction}\n\n"
    f"### Soru:\n{question}\n\n"
    f"### Cevap:\n"
)

input_ids = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **input_ids, 
    max_new_tokens=2048,
    #temperature=0.7,
    #top_p=0.9,
    #do_sample=True,
    #eos_token_id=tokenizer.eos_token_id
)

print(tokenizer.decode(outputs[0]))

Training Details

Training Data

• Dataset: arzumanabbasov/azerbaijani-gov-qa
• Language: Azerbaijani 🇦🇿
• Domain: Public services, government institutions, citizen rights, policies
• Type: Instruction-style QA pairs

Framework versions

• PEFT 0.15.2