readme: ---
  license: cc-by-4.0
  ---
  ## Aina Project's Catalan-Spanish  machine translation model for the administrative domain.
  
  ## Table of Contents
  - [Model Description](#model-description)
  - [Intended Uses and Limitations](#intended-use)
  - [How to Use](#how-to-use)
  - [Training](#training)
    - [Training data](#training-data)
    - [Training procedure](#training-procedure)
      - [Data Preparation](#data-preparation)
      - [Tokenization](#tokenization)
      - [Hyperparameters](#hyperparameters)
  - [Evaluation](#evaluation)
     - [Variable and Metrics](#variable-and-metrics)
     - [Evaluation Results](#evaluation-results)
  - [Additional Information](#additional-information)
    - [Author](#author)
    - [Contact Information](#contact-information)
    - [Copyright](#copyright)
    - [Licensing Information](#licensing-information)
    - [Funding](#funding)
    - [Disclaimer](#disclaimer)
    
  ## Model description
  
  This model is a finetuned version of [projecte-aina/mt-aina-ca-es](https://huggingface.co/projecte-aina/mt-aina-ca-es) for the administrative domain. Additionally, the model is evaluated on several public datasecomprising 5 different domains (general, adminstrative, technology, biomedical, and news).  
  
  ## Intended uses and limitations
  
  You can use this model for machine translation of administrative texts from Catalan to Spanish. 
  
  ## How to use
  
  ### Usage
  Required libraries:
  
  ```bash
  pip install ctranslate2 pyonmttok
  ```
  
  Translate a sentence using python 
  ```python
  import ctranslate2
  import pyonmttok
  from huggingface_hub import snapshot_download
  model_dir = snapshot_download(repo_id="projecte-aina/mt-aina-ca-es-adm", revision="main")
  
  tokenizer=pyonmttok.Tokenizer(mode="none", sp_model_path = model_dir + "/spm.model")
  tokenized=tokenizer.tokenize("Benvingut al projecte Aina!")
  
  translator = ctranslate2.Translator(model_dir)
  translated = translator.translate_batch([tokenized[0]])
  print(tokenizer.detokenize(translated[0][0]['tokens']))
  ```
  
  ## Training
  
  ### Training data
  
  The original model was trained on a combination of the following datasets:
  
  | Dataset           | Sentences      |
  |-------------------|----------------|
  | DOCG v2           | 8.472.786      |
  | El Periodico      | 6.483.106      |
  | EuroParl          | 1.876.669      |
  | WikiMatrix        | 1.421.077      |
  | Wikimedia         | 335.955        |
  | QED               | 71.867         |
  | TED2020 v1        | 52.177         |
  | CCMatrix v1       | 56.103.820     |
  | MultiCCAligned v1 | 2.433.418      |
  | ParaCrawl         | 15.327.808     |
  | **Total**         | **92.578.683** |
  
  This finetuned model is further trained using:
  
  | Dataset           | Sentences      |
  |-------------------|----------------|
  | AINA AAPP         | 62.773         |           
  
  ### Training procedure
  
  ### Data preparation
  
   All datasets are concatenated and filtered using the [mBERT Gencata parallel filter](https://huggingface.co/projecte-aina/mbert-base-gencata) and cleaned using the clean-corpus-n.pl script from [moses](https://github.com/moses-smt/mosesdecoder), allowing sentences between 5 and 150 words.
  
   Before training, the punctuation is normalized using a modified version of the join-single-file.py script from [SoftCatalà](https://github.com/Softcatala/nmt-models/blob/master/data-processing-tools/join-single-file.py)
  
  
  #### Tokenization
  
   All data is tokenized using sentencepiece, with 50 thousand token sentencepiece model  learned from the combination of all filtered training data. This model is included.  
  
  #### Hyperparameters
  
  The model is based on the Transformer-XLarge proposed by [Subramanian et al.](https://aclanthology.org/2021.wmt-1.18.pdf)
  The following hyperparamenters were set on the Fairseq toolkit:
  
  | Hyperparameter                     | Value                            |
  |------------------------------------|----------------------------------|
  | Architecture                       | transformer_vaswani_wmt_en_de_big|
  | Embedding size                     | 1024                             |
  | Feedforward size                   | 4096                             |
  | Number of heads                    | 16                               |
  | Encoder layers                     | 24                               |
  | Decoder layers                     | 6                                |
  | Normalize before attention         | True                             |
  | --share-decoder-input-output-embed | True                             |
  | --share-all-embeddings             | True                             |
  | Effective batch size               | 96.000                           |
  | Optimizer                          | adam                             |
  | Adam betas                         | (0.9, 0.980)                     |
  | Clip norm                          | 0.0                              |
  | Learning rate                      | 1e-3                             |
  | Lr. schedurer                      | inverse sqrt                     |
  | Warmup updates                