readme: ---
  language:
  - it
  license: apache-2.0
  tags:
  - italian
  - sequence-to-sequence
  - style-transfer
  - efficient
  - formality-style-transfer
  datasets:
  - yahoo/xformal_it
  widget:
  - text: "maronn qualcuno mi spieg' CHECCOSA SUCCEDE?!?!"
  - text: "wellaaaaaaa, ma fraté sei proprio troppo simpatiko, grazieeee!!"
  - text: "nn capisco xke tt i ragazzi lo fanno"
  - text: "IT5 è SUPERMEGA BRAVISSIMO a capire tt il vernacolo italiano!!!"
  metrics:
  - rouge
  - bertscore
  model-index:
  - name: it5-efficient-small-el32-informal-to-formal
    results:
    - task: 
        type: formality-style-transfer
        name: "Informal-to-formal Style Transfer"
      dataset:
        type: xformal_it
        name: "XFORMAL (Italian Subset)"
      metrics:
        - type: rouge1
          value: 0.430
          name: "Avg. Test Rouge1"
        - type: rouge2
          value: 0.221
          name: "Avg. Test Rouge2"
        - type: rougeL
          value: 0.408
          name: "Avg. Test RougeL"
        - type: bertscore
          value: 0.630
          name: "Avg. Test BERTScore"
  ---
  
  # IT5 Cased Small Efficient EL32 for Informal-to-formal Style Transfer 🧐
  
  *Shout-out to [Stefan Schweter](https://github.com/stefan-it) for contributing the pre-trained efficient model!*
  
  This repository contains the checkpoint for the [IT5 Cased Small Efficient EL32](https://huggingface.co/it5/it5-efficient-small-el32) model fine-tuned on Informal-to-formal style transfer on the Italian subset of the XFORMAL dataset as part of the experiments of the paper [IT5: Large-scale Text-to-text Pretraining for Italian Language Understanding and Generation](https://arxiv.org/abs/2203.03759) by [Gabriele Sarti](https://gsarti.com) and [Malvina Nissim](https://malvinanissim.github.io). 
  
  Efficient IT5 models differ from the standard ones by adopting a different vocabulary that enables cased text generation and an [optimized model architecture](https://arxiv.org/abs/2109.10686) to improve performances while reducing parameter count. The Small-EL32 replaces the original encoder from the T5 Small architecture with a 32-layer deep encoder, showing improved performances over the base model.
  
  A comprehensive overview of other released materials is provided in the [gsarti/it5](https://github.com/gsarti/it5) repository. Refer to the paper for additional details concerning the reported scores and the evaluation approach.
  
  ## Using the model
  
  Model checkpoints are available for usage in Tensorflow, Pytorch and JAX. They can be used directly with pipelines as:
  
  ```python
  from transformers import pipelines
  
  i2f = pipeline("text2text-generation", model='it5/it5-efficient-small-el32-informal-to-formal')
  i2f("nn capisco xke tt i ragazzi lo fanno")
  >>> [{"generated_text": "non comprendo perché tutti i ragazzi agiscono così"}]
  ```
  
  or loaded using autoclasses:
  
  ```python
  from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
  
  tokenizer = AutoTokenizer.from_pretrained("it5/it5-efficient-small-el32-informal-to-formal")
  model = AutoModelForSeq2SeqLM.from_pretrained("it5/it5-efficient-small-el32-informal-to-formal")
  ```
  
  If you use this model in your research, please cite our work as:
  
  ```bibtex
  @article{sarti-nissim-2022-it5,
      title={{IT5}: Large-scale Text-to-text Pretraining for Italian Language Understanding and Generation},
      author={Sarti, Gabriele and Nissim, Malvina},
      journal={ArXiv preprint 2203.03759},
      url={https://arxiv.org/abs/2203.03759},
      year={2022},
  	month={mar}
  }
  ```
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0003
  - train_batch_size: 8
  - eval_batch_size: 8
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 10.0
  
  ### Framework versions
  
  - Transformers 4.15.0
  - Pytorch 1.10.0+cu102
  - Datasets 1.17.0
  - Tokenizers 0.10.3
  