readme: ---
  language:
  - es
  - zh
  
  tags:
  - translation
  
  license: apache-2.0
  ---
  
  This model is a fine-tuned version of [Helsinki-NLP/opus-tatoeba-es-zh](https://huggingface.co/Helsinki-NLP/opus-tatoeba-es-zh) on a dataset of legal domain constructed by the author himself.
  
  
  # Intended uses & limitations
  
  This model is the result of the master graduation thesis for the Tradumatics: Translation Technologies program at the Autonomous University of Barcelona.
  
  Please refer to the GitHub repo created for this thesis for the full-text and relative open-sourced materials: https://github.com/guocheng98/MUTTT2020_TFM_ZGC
  
  The thesis intends to explain various theories and certain algorithm details about neural machine translation, thus this fine-tuned model only serves as a hands-on practice example for that objective, without any intention of productive usage.
  
  
  # Training and evaluation data
  
  The dataset is constructed from the Chinese translation of Spanish Civil Code, Spanish Constitution, and many other laws & regulations found in the database China Law Info (北大法宝 Beida Fabao), along with their source text found on Boletín Oficial del Estado and EUR-Lex.
  
  There are 9972 sentence pairs constructed. 1000 are used for evaluation and the rest for training.
  
  
  # Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 2e-05
  - train_batch_size: 8
  - eval_batch_size: 8
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_steps: 2000
  - num_epochs: 10
  - mixed_precision_training: Native AMP
  - weight_decay: 0.01
  - early_stopping_patience: 8
  
  
  # Training results
  
  Best validation loss achieved at step 5600.
  
  | Training Loss | Epoch | Step | Validation Loss |
  |:-------------:|:-----:|:----:|:---------------:|
  | 2.9584        | 0.36  | 400  | 2.6800          |
  | 2.6402        | 0.71  | 800  | 2.5017          |
  | 2.5038        | 1.07  | 1200 | 2.3907          |
  | 2.3279        | 1.43  | 1600 | 2.2999          |
  | 2.2258        | 1.78  | 2000 | 2.2343          |
  | 2.1061        | 2.14  | 2400 | 2.1961          |
  | 1.9279        | 2.5   | 2800 | 2.1569          |
  | 1.9059        | 2.85  | 3200 | 2.1245          |
  | 1.7491        | 3.21  | 3600 | 2.1227          |
  | 1.6301        | 3.57  | 4000 | 2.1169          |
  | 1.6871        | 3.92  | 4400 | 2.0979          |
  | 1.5203        | 4.28  | 4800 | 2.1074          |
  | 1.4646        | 4.63  | 5200 | 2.1024          |
  | 1.4739        | 4.99  | 5600 | 2.0905          |
  | 1.338         | 5.35  | 6000 | 2.0946          |
  | 1.3152        | 5.7   | 6400 | 2.0974          |
  | 1.306         | 6.06  | 6800 | 2.0985          |
  | 1.1991        | 6.42  | 7200 | 2.0962          |
  | 1.2113        | 6.77  | 7600 | 2.1092          |
  | 1.1983        | 7.13  | 8000 | 2.1060          |
  | 1.1238        | 7.49  | 8400 | 2.1102          |
  | 1.1417        | 7.84  | 8800 | 2.1078          |
  
  
  # Framework versions
  
  - Transformers 4.7.0
  - Pytorch 1.8.1+cu101
  - Datasets 1.8.0
  - Tokenizers 0.10.3
  