---
library_name: transformers
language:
- en
- zh
license: cc-by-4.0
base_model: Helsinki-NLP/opus-mt-zh-en
tags:
- generated_from_trainer
model-index:
- name: zhtw-en
  results: []
datasets:
- zetavg/coct-en-zh-tw-translations-twp-300k
pipeline_tag: translation
---

# zhtw-en

<details>
  <summary>English</summary>
This model translates Traditional Chinese sentences into English, with a focus on understanding Taiwanese-style Traditional Chinese and producing more accurate English translations.

This model is a fine-tuned version of [Helsinki-NLP/opus-mt-zh-en](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en) on the [zetavg/coct-en-zh-tw-translations-twp-300k](https://huggingface.co/datasets/zetavg/coct-en-zh-tw-translations-twp-300k) dataset.

It achieves the following results on the evaluation set:
- Loss: 2.4350
- Num Input Tokens Seen: 55653732

## Intended Uses & Limitations

### Intended Use Cases

- Translating single sentences from Chinese to English.
- Applications requiring understanding of the Chinese language as spoken in Taiwan.

### Limitations

- Designed for single-sentence translation so will not perform well on longer texts without pre-processing
- Sometimes hallucinates or omits information, especially with short or long inputs
- Further fine-tuning will address this

## Training and Evaluation Data

This model was trained and evaluated on the [Corpus of Contemporary Taiwanese Mandarin (COCT) translations](https://huggingface.co/datasets/zetavg/coct-en-zh-tw-translations-twp-300k) dataset.

- **Training Data:** 80% of the COCT dataset
- **Validation Data:** 20% of the COCT dataset
</details>

<details>
  <summary>Chinese</summary>
該模型旨在將繁體中文翻譯成英文，重點是理解台灣風格的繁體中文並產生更準確的英文翻譯。

模型基於 [Helsinki-NLP/opus-mt-zh-en](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en) 並在 [zetavg/coct-en-zh-tw-translations-twp-300k](https://huggingface.co/datasets/zetavg/coct-en-zh-tw-translations-twp-300k) 資料集上進行微調。

在評估集上，模型取得了以下結果：
- **損失**：2.4350
- **處理的輸入標記數量**：55,653,732

## 預期用途與限制

### 預期用途
- 將單一中文句子翻譯為英文。
- 適用於需要理解台灣中文的應用程式。

### 限制
- 本模型專為單句翻譯設計，因此在處理較長文本時可能表現不佳，若未經預處理。
- 在某些情況下，模型可能會產生幻覺或遺漏信息，特別是在輸入過短或過長的情況下。
- 進一步的微調將有助於改善這些問題。

## 訓練與評估數據

該模型使用 [當代台灣普通話語料庫 (COCT)](https://huggingface.co/datasets/zetavg/coct-en-zh-tw-translations-twp-300k) 資料集進行訓練和評估。

- **訓練資料**：COCT 資料集的 80%
- **驗證資料**：COCT 資料集的 20%
</details>

## Training Procedure

### Training Hyperparameters

The following hyperparameters were used during training:

- **Learning Rate:** 5e-05
- **Train Batch Size:** 8
- **Eval Batch Size:** 8
- **Seed:** 42
- **Optimizer:** adamw\_torch with betas=(0.9,0.999) and epsilon=1e-08
- **LR Scheduler Type:** linear
- **Number of Epochs:** 3.0

### Training Results

<details>
<summary>Click here to see the training and validation losses</summary>

| Training Loss | Epoch  | Step  | Validation Loss | Input Tokens Seen |
|:-------------:|:------:|:-----:|:---------------:|:-----------------:|
| 3.2254        | 0.0804 | 2500  | 2.9105          | 1493088           |
| 3.0946        | 0.1608 | 5000  | 2.8305          | 2990968           |
| 3.0473        | 0.2412 | 7500  | 2.7737          | 4477792           |
| 2.9633        | 0.3216 | 10000 | 2.7307          | 5967560           |
| 2.9355        | 0.4020 | 12500 | 2.6843          | 7463192           |
| 2.9076        | 0.4824 | 15000 | 2.6587          | 8950264           |
| 2.8714        | 0.5628 | 17500 | 2.6304          | 10443344          |
| 2.8716        | 0.6433 | 20000 | 2.6025          | 11951096          |
| 2.7989        | 0.7237 | 22500 | 2.5822          | 13432464          |
| 2.7941        | 0.8041 | 25000 | 2.5630          | 14919424          |
| 2.7692        | 0.8845 | 27500 | 2.5497          | 16415080          |
| 2.757         | 0.9649 | 30000 | 2.5388          | 17897832          |
| 2.7024        | 1.0453 | 32500 | 2.6006          | 19384812          |
| 2.7248        | 1.1257 | 35000 | 2.6042          | 20876844          |
| 2.6764        | 1.2061 | 37500 | 2.5923          | 22372340          |
| 2.6854        | 1.2865 | 40000 | 2.5793          | 23866100          |
| 2.683         | 1.3669 | 42500 | 2.5722          | 25348084          |
| 2.6871        | 1.4473 | 45000 | 2.5538          | 26854100          |
| 2.6551        | 1.5277 | 47500 | 2.5443          | 28332612          |
| 2.661         | 1.6081 | 50000 | 2.5278          | 29822156          |
| 2.6497        | 1.6885 | 52500 | 2.5266          | 31319476          |
| 2.6281        | 1.7689 | 55000 | 2.5116          | 32813220          |
| 2.6067        | 1.8494 | 57500 | 2.5047          | 34298052          |
| 2.6112        | 1.9298 | 60000 | 2.4935          | 35783604          |
| 2.5207        | 2.0102 | 62500 | 2.4946          | 37281092          |
| 2.4799        | 2.0906 | 65000 | 2.4916          | 38768588          |
| 2.4727        | 2.1710 | 67500 | 2.4866          | 40252972          |
| 2.4719        | 2.2514 | 70000 | 2.4760          | 41746300          |
| 2.4738        | 2.3318 | 72500 | 2.4713          | 43241188          |
| 2.4629        | 2.4122 | 75000 | 2.4630          | 44730244          |
| 2.4524        | 2.4926 | 77500 | 2.4575          | 46231060          |
| 2.435         | 2.5730 | 80000 | 2.4553          | 47718964          |
| 2.4621        | 2.6534 | 82500 | 2.4475          | 49209724          |
| 2.4492        | 2.7338 | 85000 | 2.4440          | 50712980          |
| 2.4536        | 2.8142 | 87500 | 2.4394          | 52204380          |
| 2.4148        | 2.8946 | 90000 | 2.4360          | 53695620          |
| 2.4243        | 2.9750 | 92500 | 2.4350          | 55190020          |

</details>

### Framework Versions

- Transformers 4.48.1
- Pytorch 2.3.0+cu121
- Datasets 3.2.0
- Tokenizers 0.21.0