README.md

---
base_model: Qwen/Qwen3-4B
tags:
- text-generation-inference
- transformers
- unsloth
- Qwen/Qwen3-4B
- trl
- islamic-qa
- chain-of-thought
license: apache-2.0
language:
- en
datasets:
- musaoc/Quran-reasoning-SFT
---

# Qwen3-4B-Quran-LoRA-Fine-Tuned

- **Developed by:** Ellbendls
- **License:** apache-2.0
- **Fine-tuned from:** Qwen/Qwen3-4B
- **Training Dataset:** musaoc/Quran-reasoning-SFT

A fine-tuned version of Qwen3-4B specifically designed for Islamic Q&A with chain-of-thought reasoning capabilities. This model can provide detailed explanations about Islamic concepts, jurisprudence, theology, and Quranic interpretations.

This qwen3 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)

## Model Features

- **Chain-of-Thought Reasoning**: Shows detailed thinking process before providing answers
- **Islamic Knowledge**: Specialized in Islamic concepts, jurisprudence, and theology
- **Flexible Output**: Can enable/disable thinking process based on user preference
- **Streaming Support**: Real-time response generation

## Quick Start

### Installation

```bash
pip install transformers torch soundfile ipython bitsandbytes
```

### Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM, TextStreamer
import torch

# Load the fine-tuned model
model_id = "Ellbendls/Qwen3-4b-Quran-LoRA-Fine-Tuned"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

# Ask a question
messages = [
    {"role" : "user", "content" : "What is Sabr?"},
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize = False,
    add_generation_prompt = True, # Must add for generation
    enable_thinking = True, # Disable or Enable thinking
)

from transformers import TextStreamer
_ = model.generate(
    **tokenizer(text, return_tensors = "pt").to("cuda"),
    max_new_tokens = 256, # Increase for longer outputs!
    temperature = 0.7, top_p = 0.8, top_k = 20, # For non thinking
    streamer = TextStreamer(tokenizer, skip_prompt = True),
)
```

## Parameters

- `enable_thinking = True` - Shows the model's reasoning process
- `enable_thinking = False` - Only shows the final answer
- `max_new_tokens` - Controls response length (default: 256)
- `temperature` - Controls randomness (0.1-1.0, default: 0.7)

## Example Output

```
<think>
Sabr is a crucial concept in Islam that means patience, perseverance, and endurance in the face of difficulties. It is one of the essential pillars of faith, as emphasized in the Quran. Sabr is not just about enduring hardship but also about maintaining faith and trust in Allah's plan.
</think>

Sabr is a vital virtue in Islam that teaches patience and perseverance in the face of challenges.
```

## Training Details

- **Training Method**: LoRA (Low-Rank Adaptation) fine-tuning
- **Dataset**: 857 examples from musaoc/Quran-reasoning-SFT
- **Training Steps**: 30 (demonstration model)
- **LoRA Configuration**: 
  - Rank: 32
  - Alpha: 32
  - Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- **Batch Size**: 2 per device with gradient accumulation steps of 4
- **Learning Rate**: 2e-4
- **Optimizer**: adamw_8bit

## Limitations

- **Religious Guidance**: This model provides educational information about Islamic concepts but should not be considered as official religious guidance
- **Scholarly Consultation**: For important religious matters, always consult qualified Islamic scholars
- **Training Data**: Model responses are based on training data and may not cover all Islamic perspectives

## Citation

```bibtex
@misc{qwen3-4b-quran-lora,
  title={Qwen3-4B-Quran-LoRA-Fine-Tuned},
  author={Ellbendls},
  year={2025},
  howpublished={\url{https://huggingface.co/Ellbendls/Qwen3-4b-Quran-LoRA-Fine-Tuned}},
}
```

---

*This model is designed for educational and research purposes. For authoritative Islamic guidance, please consult qualified scholars and authentic Islamic sources.*