readme: ---
  language: 
  - ca 
  - es
  - en
  tags:
  - translation
  ---
  
  ### Preprocessing
  1. Normalisation and tokenisation with moses scripts
  2. truecased with model docgWP.tcmodel.[LAN] and moses scripts
  3. bped with model model.caesen40k.bpe and subword-nmt
  - Note: no prepended tag for multilinguality
  
  ### Training Data
  1. Bilingual es-ca: DOGC, Wikimatrix, OpenSubtitles, JW300, GlobalVoices
  * Bilingual es-ca: Translations using systems trained with 1. of Oscar and Wikipedia
  2. Bilingual es-en, ca-en: United Nations, Europarl, Wikimatrix, OpenSubtitles, JW300
  * Bilingual es-en, ca-en: Translations using systems trained with 1. of the missing pairs
  
  - Final training data size for the ca/es-en: 44M parallel sentences
  - Finetuned with 1.5M real parallel data (without backtranslations)
  
  ### Model
  Transformer big with guided alignments. Relevant parameters:
  
  --beam-size 6 
  
  --normalize 0.6 
  
  --enc-depth 6  --dec-depth 6  --transformer-heads 8
  
  --transformer-preprocess n  --transformer-postprocess da 
  
  --transformer-dropout 0.1 
  
  --label-smoothing 0.1 
  
  --dim-emb 1024  --transformer-dim-ffn 4096 
  
  --transformer-dropout-attention 0.1 
  
  --transformer-dropout-ffn 0.1 
  
  --learn-rate 0.00015 --lr-warmup 8000 --lr-decay-inv-sqrt 8000 
  
  --optimizer-params 0.9 0.998 1e-09 
  
  --clip-norm 5 
  
  --tied-embeddings 
  
  --exponential-smoothing 
  
  --transformer-guided-alignment-layer 1 --guided-alignment-cost mse --guided-alignment-weight 0.1
  
  
  ## Evaluation
  
  ### Test set
  
  https://github.com/PLXIV/Gebiotoolkit/tree/master/gebiocorpus_v2
  
  ### ca2en
   BLEU|#:1|bs:1000|rs:12345|c:mixed|e:no|tok:13a|s:exp|v:2.0.0 = 47.8 (μ = 47.8 ± 0.9)
  
   chrF|#:1|bs:1000|rs:12345|c:mixed|e:yes|nc:6|nw:0|s:no|v:2.0.0 = 69.9 (μ = 69.9 ± 0.7)
  
  ### es2en
  BLEU|#:1|bs:1000|rs:12345|c:mixed|e:no|tok:13a|s:exp|v:2.0.0 = 48.9 (μ = 48.9 ± 0.9) 
  
  chrF2|#:1|bs:1000|rs:12345|c:mixed|e:yes|nc:6|nw:0|s:no|v:2.0.0 = 70.5 (μ = 70.5 ± 0.7)
  
  