README.md

---
license: mit
tags:
  - qwen
  - fine-tuning
  - LoRA
  - flask
  - instruction-tuning
  - PEFT
  - qwen3
language: en
base_model: Qwen/Qwen3-0.6B-Base
library_name: transformers
model-index:
  - name: Qwen3-Flask Full Fine-Tuned Model
    results: []
---
# 🔧 Qwen3-Flask Full Fine-Tuned Model (Merged LoRA Adapter)

This is a **fully merged fine-tuned model** based on [Qwen/Qwen3-0.6B-Base](https://huggingface.co/Qwen/Qwen3-0.6B-Base). It was trained on a rich developer-focused Q&A dataset covering Flask internals. Fine-tuning was done using LoRA (Low-Rank Adaptation) and later merged into the base model for ease of deployment.

---

## 🧠 Project Objective

Flask’s documentation, while comprehensive, often lacks developer-centric summaries or Q&A-style explanations. This project bridges that gap by:

- Turning **raw Flask source code** and docstrings into **instructional Q&A data**
- Fine-tuning a strong open LLM (Qwen) to produce **developer-style responses**
- Providing both **LoRA and full-weight versions** for flexible deployment

---

## 📚 Use Cases

- 🔍 Explaining internal APIs and decorators (`before_request`, `url_defaults`, etc.)
- 🔄 Clarifying Flask upgrade/migration behaviors
- 📘 Summarizing docstring-heavy logic in natural Q&A form
- ⚙️ Assisting junior devs learning Flask internals

---

## 🧪 Dataset Creation

A custom script extracted:
- All functions, classes, methods, and docstrings from the Flask codebase (`.py` files)
- Filtered to 345 valid logic-rich chunks out of 804 total
- Each chunk was passed to Gemini using a Q&A generation prompt

**Total Q&A pairs generated**: `1425`

Example:
```json
{
  "instruction": "What does `before_request` do in Flask?",
  "input": "This function runs before each request, useful for checking login sessions, etc.",
  "output": "`before_request` is a Flask decorator used to register a function that runs before each request. It is commonly used to implement access control logic or session checks."
}

## 🧪 Fine-Tuning Details

- **Model**: Qwen/Qwen3-0.6B-Base
- **PEFT Type**: LoRA (r=8, alpha=16)
- **Quantization**: 4-bit NF4 using `bitsandbytes`
- **Training Library**: `transformers`, `peft`, `datasets`
- **Device**: Single NVIDIA RTX 3060 6GB VRAM (consumer laptop)
- **Dataset**: 1000+ cleaned Q&A pairs from Flask official documentation

---

## 🧠 Prompt Format

The model was fine-tuned on Alpaca-style prompts:

```text
### Instruction:
<What do you want to know?>

### Input:
<Any supporting context>

### Response:
<Model-generated answer>
```

## 🏗️ Training

Used PEFT + LoRA with:

- Rank: 16
- Alpha: 32
- Target modules: query_key_value
- Epochs: 3
- Dataset: 1425 Q&A JSONL entries

## 📊 Evaluation

- Evaluated using BLEU, ROUGE, and manual inspection.
- Improved consistency and structured response formatting for framework-specific queries.

## 🔍 Inference

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("devanshdhir/qwen3-flask-full", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("devanshdhir/qwen3-flask-full", trust_remote_code=True)

prompt = """### Instruction:
What is the purpose of `url_defaults` in Flask?
### Input:
Related excerpt from docs...
### Response:"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=300)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

```