heba1998/Qwen-LoRA-Estate

Qwen3 0.6B - Real Estate Fine-Tuned Adapter (LoRA)

This model is a Qwen3-0.6B adapter fine-tuned using LoRA on a real estate dataset for tasks such as property description generation and value estimation. Fine-tuning was performed using LLaMA Factory.

Base Model

• Qwen/Qwen3-0.6B
• Fine-tuned using LoRA with lora_rank=64 targeting all transformer layers.

Fine-Tuning Details

Setting	Value
Framework	LLaMA Factory
Finetuning Type	LoRA
LoRA Rank	64
Dataset	Custom real estate dataset
Cutoff Length	3500 tokens
Epochs	3
Batch Size	1 (accumulated over 8 steps)
Learning Rate	1e-4
Scheduler	Cosine
Evaluation Metric	eval_loss
Best Model Criterion	Lowest validation loss

Dataset

• Sample LlaMa-Factory Dataset : heba1998/real-estate-data-for-llm-fine-tuning
• Full Dataset: heba1998/real-estate-data-for-llm-fine-tuning

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B", device_map="auto", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B", trust_remote_code=True)

adapter = PeftModel.from_pretrained(base_model, "heba1998/Qwen-LoRA-Estate")