Bryan
commited on
Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,68 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# 小參數長鏈思考模型(Chain-of-Thought for Traditional Chinese)
|
2 |
+
Finetuned LLaMA 3.2 3B on Chain-of-Thought Reasoning in Traditional Chinese
|
3 |
+
|
4 |
+
## 模型簡介 | Model Overview
|
5 |
+
|
6 |
+
這是一個專為繁體中文社群設計的開源長鏈思考(Chain-of-Thought, CoT)微調模型,基於 Meta LLaMA 3.2 3B 架構進行微調,聚焦於增強模型在繁體中文語境下的推理能力與多步邏輯表達。
|
7 |
+
|
8 |
+
This is an open-source Chain-of-Thought (CoT) finetuned model for the Traditional Chinese community, built upon Meta's LLaMA 3.2 3B architecture. It enhances multi-step reasoning and logical articulation within a Traditional Chinese context.
|
9 |
+
|
10 |
+
## 訓練動機 | Training Motivation
|
11 |
+
|
12 |
+
作為一名人工智慧愛好者,我發現目前針對繁體中文語境的長鏈思考模型仍然稀缺,許多開源模型偏向英文或簡體中文。因此,我著手打造此模型,期望為繁體中文用戶提供更合適的邏輯推理基礎模型,並推廣 CoT 技術在華語世界的應用與理解。
|
13 |
+
|
14 |
+
As an AI enthusiast, I noticed the scarcity of open-source CoT models tailored for Traditional Chinese. Most models are either English-based or optimized for Simplified Chinese. This project aims to fill that gap, offering a dedicated reasoning-friendly model for Traditional Chinese users, and promoting CoT applications in the broader Chinese-speaking world.
|
15 |
+
|
16 |
+
## 特性簡述 | Key Features
|
17 |
+
|
18 |
+
- 語言支援:專為繁體中文設計,保留文化語感
|
19 |
+
- 推理能力:優化多步邏輯思考與問題拆解
|
20 |
+
- 開源導向:歡迎社群參與微調與改進
|
21 |
+
- 小參數模型:3B 規模,適合在中等資源設備上運行
|
22 |
+
|
23 |
+
- Language Support: Tuned for Traditional Chinese with cultural nuance
|
24 |
+
- Reasoning-Ready: Enhanced for multi-step problem-solving
|
25 |
+
- Open-Source Friendly: Community contributions are welcome
|
26 |
+
- Lightweight: 3B size, ideal for moderate hardware environments
|
27 |
+
|
28 |
+
## 訓練細節 | Training Details
|
29 |
+
|
30 |
+
- Base Model:Meta LLaMA 3.2 3B
|
31 |
+
- 微調任務:Chain-of-Thought prompting in Traditional Chinese
|
32 |
+
- 資料集來源:自建與繁體化處理的開源數據(涵蓋數學、邏輯推理、日常問答等)
|
33 |
+
- 訓練策略:使用 LoRA 精簡微調技術(Low-Rank Adaptation),提升上下文理解與推理連貫性
|
34 |
+
- 硬體資源:單張 NVIDIA RTX 4060,進行約 16 小時微調
|
35 |
+
- 訓練框架:基於 Hugging Face Transformers + PEFT + bitsandbytes 訓練
|
36 |
+
|
37 |
+
### Training Details (English)
|
38 |
+
|
39 |
+
- Base Model: Meta LLaMA 3.2 3B
|
40 |
+
- Task: Chain-of-Thought prompting in Traditional Chinese
|
41 |
+
- Dataset: Custom-built and adapted Traditional Chinese datasets (math, logical reasoning, daily QA)
|
42 |
+
- Tuning Strategy: Lightweight LoRA finetuning to boost context handling and step-by-step reasoning
|
43 |
+
- Hardware: Trained on a single NVIDIA RTX 4060 GPU for approximately 16 hours
|
44 |
+
- Framework: Powered by Hugging Face Transformers, PEFT, and bitsandbytes
|
45 |
+
|
46 |
+
## 使用建議 | Usage Tips
|
47 |
+
|
48 |
+
此模型適用於以下應用場景:
|
49 |
+
- 推理任務與數學解題
|
50 |
+
- 教學與邏輯問答
|
51 |
+
- 多步驟任務規劃
|
52 |
+
|
53 |
+
適合與 CoT 提示語搭配,例如:
|
54 |
+
「請一步一步解釋你的推理過程。」
|
55 |
+
|
56 |
+
Recommended for tasks such as:
|
57 |
+
- Logical reasoning and math problems
|
58 |
+
- Educational QA
|
59 |
+
- Step-by-step task planning
|
60 |
+
|
61 |
+
Pair well with CoT-style prompts like:
|
62 |
+
"Please explain your reasoning step by step."
|
63 |
+
|
64 |
+
## 歡迎貢獻 | Contribute
|
65 |
+
|
66 |
+
此模型開放給社群一同優化與實驗,若你也關心繁體中文在 AI 領域的表現,歡迎 fork、finetune 或提交 PR。
|
67 |
+
|
68 |
+
This model is open to community collaboration. If you care about advancing Traditional Chinese capabilities in AI, feel free to fork, finetune, or open a PR!
|