Model Card for Reddit Finance SFT

A fine-tuned language model specifically designed to understand and respond to financial discussions in a Reddit-style conversational manner. This model has been trained on finance-related Reddit data to better engage with financial topics, questions, and analysis.

Model Description

This is a supervised fine-tuned (SFT) version of the Qwen3-0.6B model, specifically adapted for financial discussions and Reddit-style conversations. The model has been trained using LoRA (Low-Rank Adaptation) on curated Reddit finance data to improve its ability to engage with financial topics in a conversational, community-driven manner.

Developed by: https://huggingface.co/BoostedJonP
Model type: Causal Language Model
Language(s) (NLP): English
Finetuned from model: Qwen/Qwen3-0.6B
Training method: Supervised Fine-Tuning (SFT) with LoRA

Model Sources

Repository: Repository URL
Base Model: Qwen/Qwen3-0.6B

Direct Use

This model is designed for:

Financial discussion and analysis in a conversational style
Responding to Reddit-style financial questions and discussions
Providing financial insights in a community-driven manner
Engaging with finance-related topics in a casual, accessible way

How to Get Started with the Model

Generic text generation

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "Qwen/Qwen3-0.6B"
adapter_model = "BoostedJonP/Qwen3-0.6B-finance-reddit-sft"

tokenizer = AutoTokenizer.from_pretrained(base_model)
base_model = AutoModelForCausalLM.from_pretrained(base_model, device_map="auto", torch_dtype="auto")
model = PeftModel.from_pretrained(base_model, adapter_model)

prompt = "What's a good investment strategy for a student?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output_ids = model.generate(**inputs, max_new_tokens=100)
output_text = tokenizer.decode(output_ids[0], skip_special_tokens=True)
print(output_text)

Chat usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "Qwen/Qwen3-0.6B"
adapter_model = "BoostedJonP/Qwen3-0.6B-finance-reddit-sft"

tokenizer = AutoTokenizer.from_pretrained(base_model)
base_model = AutoModelForCausalLM.from_pretrained(base_model, device_map="auto", torch_dtype="auto")
model = PeftModel.from_pretrained(base_model, adapter_model)

prompt = "What's a good investment strategy for a student?"
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

output_ids = model.generate(**model_inputs, max_new_tokens=100)

content = tokenizer.decode(output_ids[0], skip_special_tokens=True)
print(content)

Training Details

Training Data

The model was trained on the winddude/reddit_finance_43_250k dataset, which contains posts from various finance-related Reddit subreddits including:

r/wallstreetbets
r/investing
r/stocks
r/personalfinance
r/options
r/cryptocurrency
And many other finance-related communities

Data Preprocessing:

Filtered out low-quality posts using pattern matching
Removed URLs and user mentions for cleaner training data
Formatted data into instruction-response pairs
Applied text cleaning and normalization

Training Procedure

Preprocessing

Data Filtering: Removed posts matching predefined low-quality patterns
Text Cleaning: Removed URLs, user mentions, and other noise
Formatting: Converted to instruction-response format for SFT
Tokenization: Used Qwen tokenizer for consistent encoding

Training Hyperparameters

Training regime: FP16 mixed precision
Base model: Qwen/Qwen3-0.6B
LoRA rank (r): 8
LoRA alpha: 32
Target modules: q_proj, k_proj, v_proj, o_proj
Dropout: 0.05
Batch size: 4 per device
Learning rate: 2e-4
Weight decay: 0.01
Warmup ratio: 0.03
Training epochs: 5

Speeds, Sizes, Times

Model size: ~0.6B parameters (base) + LoRA adapters

BoostedJonP
/

Qwen3-0.6B-finance-reddit-sft