Model Card for Llama-3.2-3B-F1-Instruct

Note: The checkpoint for this model will be released soon. Please stay tuned. 🙏

Llama-3.2-3B-F1-Instruct 是由 Twinkle AI 與 APMIC 合作開發，並在國家高速網路與計算中心技術指導之下，針對中華民國台灣語境與任務需求所微調之繁體中文語言模型，涵蓋法律、教育、生活應用等多元場景，並以高指令跟隨能力為目標進行強化。

Model Details

Model Description

• Developed by: Huang Liang Hsun、Min Yi Chen、Wen Bin Lin、Chao Chun Chuang & Dave Sung (All authors have contributed equally to this work.)
• Funded by: APMIC
• Model type: LlamaForCausalLM
• Language(s) (NLP): Tranditional Chinese & English
• License: llama3.2

Model Sources

• Repository: twinkle-ai/Llama-3.2-3B-F1-Instruct
• Paper: (TBA)

Evaluation

Results

下表採用 🌟 Twinkle Eval 評測框架

模型	評測模式	TMMLU+(%)	台灣法律(%)	MMLU(%)	測試次數	選項排序
mistralai/Mistral-Small-24B-Instruct-2501	box	56.15 (±0.0172)	37.48 (±0.0098)	74.61 (±0.0154)	3	隨機
meta-llama/Llama-3.2-3B-Instruct	box	15.49 (±0.0104)	25.68 (±0.0200)	6.90 (±0.0096)	3	隨機
meta-llama/Llama-3.2-3B-Instruct	pattern	35.85 (±0.0174)	32.22 (±0.0023)	59.33 (±0.0168)	3	隨機
MediaTek-Research/Llama-Breeze2-3B-Instruct	pattern	40.32 (±0.0181)	38.92 (±0.0193)	55.37 (±0.0180)	3	隨機
twinkle-ai/Llama-3.2-3B-F1-Instruct (ours)	box	42.18 (±0.0197)	31.26 (±0.0354)	52.07 (±0.0189)	3	隨機

下表用 lighteval 評測框架

模型	MATH-500	GPQA Diamond
meta-llama/Llama-3.2-3B-Instruct	44.40	27.78
twinkle-ai/Llama-3.2-3B-F1-Instruct (ours)	51.40	33.84

Citation

@misc{twinkleai2025llama3.2f1,
  title        = {Llama-3.2-3B-F1-Instruct: A Traditional Chinese Instruction-Tuned Language Model for Taiwan},
  author       = {Huang, Liang Hsun and Chen, Min Yi and Lin, Wen Bin and Chuang, Chao Chun and Sung, Dave},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/twinkle-ai/Llama-3.2-3B-F1-Instruct}},
  note         = {Twinkle AI and APMIC. All authors contributed equally.}
}

Acknowledge

特此感謝國家高速網路與計算中心的指導與 APMIC 的算力支援，才得以讓本專案訓利完成。

Model Card Authors

Twinkle AI

Model Card Contact

Twinkle AI