|
|
--- |
|
|
license: artistic-2.0 |
|
|
language: |
|
|
- zh |
|
|
- en |
|
|
tags: |
|
|
- chain-of-thought |
|
|
- llama |
|
|
- traditional-chinese |
|
|
- reasoning |
|
|
- lora |
|
|
library_name: transformers |
|
|
datasets: |
|
|
- openai/gsm8k |
|
|
pipeline_tag: text-generation |
|
|
model_creator: BryanADA |
|
|
base_model: |
|
|
- meta-llama/Llama-3.2-3B-Instruct |
|
|
--- |
|
|
|
|
|
# 小參數長鏈思考模型（Chain-of-Thought for Traditional Chinese） |
|
|
Finetuned LLaMA 3.2 3B on Chain-of-Thought Reasoning in Traditional Chinese |
|
|
|
|
|
## 模型簡介 | Model Overview |
|
|
|
|
|
這是一個專為繁體中文社群設計的開源長鏈思考（Chain-of-Thought, CoT）微調模型，基於 Meta LLaMA 3.2 3B 架構進行微調，聚焦於增強模型在繁體中文語境下的推理能力與多步邏輯表達。 |
|
|
|
|
|
This is an open-source Chain-of-Thought (CoT) finetuned model for the Traditional Chinese community, built upon Meta's LLaMA 3.2 3B architecture. It enhances multi-step reasoning and logical articulation within a Traditional Chinese context. |
|
|
|
|
|
## 訓練動機 | Training Motivation |
|
|
|
|
|
作為一名人工智慧愛好者，我發現目前針對繁體中文語境的長鏈思考模型仍然稀缺，許多開源模型偏向英文或簡體中文。因此，我著手打造此模型，期望為繁體中文用戶提供更合適的邏輯推理基礎模型，並推廣 CoT 技術在華語世界的應用與理解。 |
|
|
|
|
|
As an AI enthusiast, I noticed the scarcity of open-source CoT models tailored for Traditional Chinese. Most models are either English-based or optimized for Simplified Chinese. This project aims to fill that gap, offering a dedicated reasoning-friendly model for Traditional Chinese users, and promoting CoT applications in the broader Chinese-speaking world. |
|
|
|
|
|
## 特性簡述 | Key Features |
|
|
|
|
|
- 語言支援：專為繁體中文設計，保留文化語感 |
|
|
- 推理能力：優化多步邏輯思考與問題拆解 |
|
|
- 開源導向：歡迎社群參與微調與改進 |
|
|
- 小參數模型：3B 規模，適合在中等資源設備上運行 |
|
|
|
|
|
- Language Support: Tuned for Traditional Chinese with cultural nuance |
|
|
- Reasoning-Ready: Enhanced for multi-step problem-solving |
|
|
- Open-Source Friendly: Community contributions are welcome |
|
|
- Lightweight: 3B size, ideal for moderate hardware environments |
|
|
|
|
|
## 訓練細節 | Training Details |
|
|
|
|
|
- Base Model：Meta LLaMA 3.2 3B |
|
|
- 微調任務：Chain-of-Thought prompting in Traditional Chinese |
|
|
- 資料集來源：自建與繁體化處理的開源數據（涵蓋數學、邏輯推理、日常問答等） |
|
|
- 訓練策略：使用 LoRA 精簡微調技術（Low-Rank Adaptation），提升上下文理解與推理連貫性 |
|
|
- 硬體資源：單張 NVIDIA RTX 4060，進行約 16 小時微調 |
|
|
- 訓練框架：基於 Hugging Face Transformers + PEFT + bitsandbytes 訓練 |
|
|
|
|
|
### Training Details (English) |
|
|
|
|
|
- Base Model: Meta LLaMA 3.2 3B |
|
|
- Task: Chain-of-Thought prompting in Traditional Chinese |
|
|
- Dataset: Custom-built and adapted Traditional Chinese datasets (math, logical reasoning, daily QA) |
|
|
- Tuning Strategy: Lightweight LoRA finetuning to boost context handling and step-by-step reasoning |
|
|
- Hardware: Trained on a single NVIDIA RTX 4060 GPU for approximately 16 hours |
|
|
- Framework: Powered by Hugging Face Transformers, PEFT, and bitsandbytes |
|
|
|
|
|
## 使用建議 | Usage Tips |
|
|
|
|
|
此模型適用於以下應用場景： |
|
|
- 推理任務與數學解題 |
|
|
- 教學與邏輯問答 |
|
|
- 多步驟任務規劃 |
|
|
|
|
|
適合與 CoT 提示語搭配，例如： |
|
|
「請一步一步解釋你的推理過程。」 |
|
|
|
|
|
Recommended for tasks such as: |
|
|
- Logical reasoning and math problems |
|
|
- Educational QA |
|
|
- Step-by-step task planning |
|
|
|
|
|
Pair well with CoT-style prompts like: |
|
|
"Please explain your reasoning step by step." |
|
|
|
|
|
## 歡迎貢獻 | Contribute |
|
|
|
|
|
此模型開放給社群一同優化與實驗，若你也關心繁體中文在 AI 領域的表現，歡迎 fork、finetune 或提交 PR。 |
|
|
|
|
|
This model is open to community collaboration. If you care about advancing Traditional Chinese capabilities in AI, feel free to fork, finetune, or open a PR! |
