readme: ---
  license: apache-2.0
  ---
  ## Lutech-AI/I-SPIn
  **I**talian-**S**entence **P**air **In**ference, AKA **I-SPIn**.<br>
  This is a fine-tuned version of the model [paraphrase-multilingual-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2).<br>
  Its main task is to perform the [Natural Language Inference (NLI)](https://nlp.stanford.edu/projects/snli/) task in the Italian language.<br>
  The prediction labels may assume three possible values: 
  1. 1 means the model predicts <em>entailment</em>;
  2. 0 represents the <em>neutral</em> case;
  3. -1 corresponds to <em>contradiction</em>.
   
  ## How it was trained
  1. Train [paraphrase-multilingual-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2) on the NLI task;
  2. Apply Knowledge Distillation on the output of (1.) with IT-EN translation dataset to retain NLI knowledge and improve Italian language comprehension.
  
  # Usage #1 (HuggingFace Transformers)
  In the environment on which you want to run the project, type:
  ```markdown
  pip install --extra-index-url https://test.pypi.org/simple/ ispin
  ```
  
  NOTE: during the first execution, a total of two different models will be downloaded: 
  1. I-SPIn;
  2. paraphrase-multilingual-mpnet-base-v2.
  
  Each is roughly 1GB in dimension.
  
  ## Retrieve embeddings
  If you installed the package correctly, you can retrieve embeddings in the following way:
  ```python
  from ispin.ISPIn import ISPIn
  model = ISPIn.from_pretrained('Lutech-AI/I-SPIn')
  sentences = ['Questa è una frase di prova', 'Testando il funzionamento del modello']
  sentence_embeddings = model(sentences)
  print(sentence_embeddings) # -> torch.Size(2, 768)
  ```
  
  ## Retrieve labels
  If you installed the package correctly, you can retrieve labels in the following way:
  ```python
  from ispin.ISPIn import ISPIn
  model = ISPIn.from_pretrained('Lutech-AI/I-SPIn')
  premises = ['Il modello sta funzionando correttamente', 'Il modello non funziona correttamente']
  hypothesis = ['Testando il funzionamento del modello']
  premises_embeddings = model(premises)
  hypothesis_embeddings = model(hypothesis)
  predictions = model.predict(
      premises_embeddings,
      hypothesis_embeddings,
      one_to_many = False
  )
  print(predictions) # -> [0 -1]
  ```
  
  The computation is subdivided in two tasks (embedding, classification) to simplify a custom fine-tuning process.
  
  If you want to further optimize this classification head, you might want to deepcopy the layers and continue training 
  (one can choose which layers by slicing the list):
  ```python
  import torch
  import copy
  module_list = torch.nn.ModuleList(list(copy.deepcopy(model.layers))[start:end])
  ```
  
  # Usage #2 (cloning repo) (will be deleted)
  In a terminal located in your project folder, type: 'git clone https://huggingface.co/Lutech-AI/I-SPIn/ ISPIn'. <br>
  Please specify the final 'ISPIn' to avoid complications when calling the Python module. <br>
  Then, in the code where you call the model, substitute the line:
  ```python
  model = ISPIn.from_pretrained('Lutech-AI/I-SPIn')
  ```
  with:
  ```python
  model = ISPIn.from_pretrained('[your/path]/I-SPIn')
  ```
  
  ## Full model architecture
  ```markdown
  ISPIn(
    (encoder): XLMRobertaModel(...) # transformers internal implementation of 'paraphrase-multilingual-mpnet-base-v2'
    (layers): ModuleList(
      (0): Linear(in_features=1536, out_features=1024, bias=True)
      (1): Linear(in_features=1024, out_features=512, bias=True)
      (2): Linear(in_features=512, out_features=256, bias=True)
      (3): Linear(in_features=256, out_features=128, bias=True)
      (4): Linear(in_features=128, out_features=64, bias=True)
      (5): Linear(in_features=64, out_features=3, bias=True)
    )
    (activation): GELU()
  )
  ```
  
  ## Evaluation results
  |                 Dataset                | Metric       | Performance |
  |:--------------------------------------:|--------------|-------------|
  | [RTE3-ITA](https://github.com/gilnoh/RTEFormatWork/tree/master/RTE3-ITdata-original-format)                               | Accuracy     | 68%         |
  | [RTE3-ITA](https://github.com/gilnoh/RTEFormatWork/tree/master/RTE3-ITdata-original-format)                               | Min F1-Score | 60%         |
  | [RTE-2009-ITA](https://live.european-language-grid.eu/catalogue/corpus/8121/download/)                             | Accuracy     | 59%         |
  | [RTE-2009-ITA](https://live.european-language-grid.eu/catalogue/corpus/8121/download/)                               | Min F1-Score | 31%         |
  | [SNLI](https://nlp.stanford.edu/projects/snli/) (IT) translated w/[NLLB-600M](https://huggingface.co/facebook/nllb-200-distilled-600M)            | Accuracy     | 74%         |
  | [MNLI-Matched](https://cims.nyu.edu/~sbowman/multinli/) (IT) translated w/[NLLB-600M](https://huggingface.co/facebook/nllb-200-distilled-600M)    | Accuracy     | 72%         |
  | [MNLI-Mismatched](https://cims.nyu.edu/~sbowman/multinli/) (IT) translated w/[NLLB-600M](https://huggingface.co/facebook/nllb-200-distilled-600M) | 