Bryan commited on
Commit
1711144
·
verified ·
1 Parent(s): bfdf54b

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +68 -0
README.md ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 小參數長鏈思考模型(Chain-of-Thought for Traditional Chinese)
2
+ Finetuned LLaMA 3.2 3B on Chain-of-Thought Reasoning in Traditional Chinese
3
+
4
+ ## 模型簡介 | Model Overview
5
+
6
+ 這是一個專為繁體中文社群設計的開源長鏈思考(Chain-of-Thought, CoT)微調模型,基於 Meta LLaMA 3.2 3B 架構進行微調,聚焦於增強模型在繁體中文語境下的推理能力與多步邏輯表達。
7
+
8
+ This is an open-source Chain-of-Thought (CoT) finetuned model for the Traditional Chinese community, built upon Meta's LLaMA 3.2 3B architecture. It enhances multi-step reasoning and logical articulation within a Traditional Chinese context.
9
+
10
+ ## 訓練動機 | Training Motivation
11
+
12
+ 作為一名人工智慧愛好者,我發現目前針對繁體中文語境的長鏈思考模型仍然稀缺,許多開源模型偏向英文或簡體中文。因此,我著手打造此模型,期望為繁體中文用戶提供更合適的邏輯推理基礎模型,並推廣 CoT 技術在華語世界的應用與理解。
13
+
14
+ As an AI enthusiast, I noticed the scarcity of open-source CoT models tailored for Traditional Chinese. Most models are either English-based or optimized for Simplified Chinese. This project aims to fill that gap, offering a dedicated reasoning-friendly model for Traditional Chinese users, and promoting CoT applications in the broader Chinese-speaking world.
15
+
16
+ ## 特性簡述 | Key Features
17
+
18
+ - 語言支援:專為繁體中文設計,保留文化語感
19
+ - 推理能力:優化多步邏輯思考與問題拆解
20
+ - 開源導向:歡迎社群參與微調與改進
21
+ - 小參數模型:3B 規模,適合在中等資源設備上運行
22
+
23
+ - Language Support: Tuned for Traditional Chinese with cultural nuance
24
+ - Reasoning-Ready: Enhanced for multi-step problem-solving
25
+ - Open-Source Friendly: Community contributions are welcome
26
+ - Lightweight: 3B size, ideal for moderate hardware environments
27
+
28
+ ## 訓練細節 | Training Details
29
+
30
+ - Base Model:Meta LLaMA 3.2 3B
31
+ - 微調任務:Chain-of-Thought prompting in Traditional Chinese
32
+ - 資料集來源:自建與繁體化處理的開源數據(涵蓋數學、邏輯推理、日常問答等)
33
+ - 訓練策略:使用 LoRA 精簡微調技術(Low-Rank Adaptation),提升上下文理解與推理連貫性
34
+ - 硬體資源:單張 NVIDIA RTX 4060,進行約 16 小時微調
35
+ - 訓練框架:基於 Hugging Face Transformers + PEFT + bitsandbytes 訓練
36
+
37
+ ### Training Details (English)
38
+
39
+ - Base Model: Meta LLaMA 3.2 3B
40
+ - Task: Chain-of-Thought prompting in Traditional Chinese
41
+ - Dataset: Custom-built and adapted Traditional Chinese datasets (math, logical reasoning, daily QA)
42
+ - Tuning Strategy: Lightweight LoRA finetuning to boost context handling and step-by-step reasoning
43
+ - Hardware: Trained on a single NVIDIA RTX 4060 GPU for approximately 16 hours
44
+ - Framework: Powered by Hugging Face Transformers, PEFT, and bitsandbytes
45
+
46
+ ## 使用建議 | Usage Tips
47
+
48
+ 此模型適用於以下應用場景:
49
+ - 推理任務與數學解題
50
+ - 教學與邏輯問答
51
+ - 多步驟任務規劃
52
+
53
+ 適合與 CoT 提示語搭配,例如:
54
+ 「請一步一步解釋你的推理過程。」
55
+
56
+ Recommended for tasks such as:
57
+ - Logical reasoning and math problems
58
+ - Educational QA
59
+ - Step-by-step task planning
60
+
61
+ Pair well with CoT-style prompts like:
62
+ "Please explain your reasoning step by step."
63
+
64
+ ## 歡迎貢獻 | Contribute
65
+
66
+ 此模型開放給社群一同優化與實驗,若你也關心繁體中文在 AI 領域的表現,歡迎 fork、finetune 或提交 PR。
67
+
68
+ This model is open to community collaboration. If you care about advancing Traditional Chinese capabilities in AI, feel free to fork, finetune, or open a PR!