File size: 3,503 Bytes
34b91d5 1700b5a 34b91d5 05eb4ec 1700b5a 05eb4ec 34b91d5 05eb4ec 34b91d5 05eb4ec 34b91d5 05eb4ec 34b91d5 05eb4ec 34b91d5 05eb4ec |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 |
---
license: mit
tags:
- qwen
- fine-tuning
- LoRA
- flask
- instruction-tuning
- PEFT
- qwen3
language: en
base_model: Qwen/Qwen3-0.6B-Base
library_name: transformers
model-index:
- name: Qwen3-Flask Full Fine-Tuned Model
results: []
---
# 🔧 Qwen3-Flask Full Fine-Tuned Model (Merged LoRA Adapter)
This is a **fully merged fine-tuned model** based on [Qwen/Qwen3-0.6B-Base](https://huggingface.co/Qwen/Qwen3-0.6B-Base). It was trained on a rich developer-focused Q&A dataset covering Flask internals. Fine-tuning was done using LoRA (Low-Rank Adaptation) and later merged into the base model for ease of deployment.
---
## 🧠 Project Objective
Flask’s documentation, while comprehensive, often lacks developer-centric summaries or Q&A-style explanations. This project bridges that gap by:
- Turning **raw Flask source code** and docstrings into **instructional Q&A data**
- Fine-tuning a strong open LLM (Qwen) to produce **developer-style responses**
- Providing both **LoRA and full-weight versions** for flexible deployment
---
## 📚 Use Cases
- 🔍 Explaining internal APIs and decorators (`before_request`, `url_defaults`, etc.)
- 🔄 Clarifying Flask upgrade/migration behaviors
- 📘 Summarizing docstring-heavy logic in natural Q&A form
- ⚙️ Assisting junior devs learning Flask internals
---
## 🧪 Dataset Creation
A custom script extracted:
- All functions, classes, methods, and docstrings from the Flask codebase (`.py` files)
- Filtered to 345 valid logic-rich chunks out of 804 total
- Each chunk was passed to Gemini using a Q&A generation prompt
**Total Q&A pairs generated**: `1425`
Example:
```json
{
"instruction": "What does `before_request` do in Flask?",
"input": "This function runs before each request, useful for checking login sessions, etc.",
"output": "`before_request` is a Flask decorator used to register a function that runs before each request. It is commonly used to implement access control logic or session checks."
}
## 🧪 Fine-Tuning Details
- **Model**: Qwen/Qwen3-0.6B-Base
- **PEFT Type**: LoRA (r=8, alpha=16)
- **Quantization**: 4-bit NF4 using `bitsandbytes`
- **Training Library**: `transformers`, `peft`, `datasets`
- **Device**: Single NVIDIA RTX 3060 6GB VRAM (consumer laptop)
- **Dataset**: 1000+ cleaned Q&A pairs from Flask official documentation
---
## 🧠 Prompt Format
The model was fine-tuned on Alpaca-style prompts:
```text
### Instruction:
<What do you want to know?>
### Input:
<Any supporting context>
### Response:
<Model-generated answer>
```
## 🏗️ Training
Used PEFT + LoRA with:
- Rank: 16
- Alpha: 32
- Target modules: query_key_value
- Epochs: 3
- Dataset: 1425 Q&A JSONL entries
## 📊 Evaluation
- Evaluated using BLEU, ROUGE, and manual inspection.
- Improved consistency and structured response formatting for framework-specific queries.
## 🔍 Inference
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("devanshdhir/qwen3-flask-full", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("devanshdhir/qwen3-flask-full", trust_remote_code=True)
prompt = """### Instruction:
What is the purpose of `url_defaults` in Flask?
### Input:
Related excerpt from docs...
### Response:"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=300)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
|