readme: ---
  inference: false
  pipeline_tag: sentence-similarity
  tags:
  - sentence-transformers
  - feature-extraction
  - sentence-similarity
  - transformers
  ---
  
  <div style="clear: both;">
    <div style="float: left; margin-right 1em;">
      <h1><strong>FinISH (Finance-Identifying Sroberta for Hypernyms)</strong></h1>
    </div>
    <div>
      <h2><img src="https://pbs.twimg.com/profile_images/1333760924914753538/fQL4zLUw_400x400.png" alt="" width="25" height="25"></h2>
    </div>
    </div>
    
  We present FinISH, a [SRoBERTa](https://huggingface.co/sentence-transformers/nli-roberta-base-v2) base model fine-tuned on the [FIBO ontology](https://spec.edmcouncil.org/fibo/) dataset for domain-specific representation learning on the [**Semantic Search**](https://www.sbert.net/examples/applications/semantic-search/README.html) downstream task.
  
  ## SRoBERTa Model Architecture
  Sentence-RoBERTa (SRoBERTa) is a modification of the pretrained RoBERTa network that uses siamese and triplet network structures to derive semantically meaningful sentence embeddings that can be compared using cosine-similarity. This reduces the effort for finding the most similar pair from 65 hours with RoBERTa to about 5 seconds with SRoBERTa, while maintaining the accuracy from RoBERTa. SRoBERTa has been evaluated on common STS tasks and transfer learning tasks, where it outperforms other state-of-the-art sentence embeddings methods.
  
  Paper: [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/pdf/1908.10084.pdf).
  
  Authors: *Nils Reimers and Iryna Gurevych*.
  
  ## Details on the downstream task (Semantic Search for Text Classification)
  The objective of this task is to correctly classify a given term in the financial domain according to its prototypical hypernym in a list of available hypernyms:
  * Bonds
  * Forward
  * Funds
  * Future
  * MMIs (Money Market Instruments)
  * Option
  * Stocks
  * Swap
  * Equity Index
  * Credit Index
  * Securities restrictions
  * Parametric schedules
  * Debt pricing and yields
  * Credit Events
  * Stock Corporation
  * Central Securities Depository
  * Regulatory Agency
  
  This kind-based approach relies on identifying the closest hypernym semantically to the given term (even if they possess common properties with other hypernyms).
  
  #### Data Description
  The data is a scraped list of term definitions from the FIBO ontology website where each definition has been mapped to its closest hypernym from the proposed labels.
  For multi-sentence definitions, we applied sentence-splitting by punctuation delimiters. We also applied lowercase transformation on all input data.
  
  #### Data Instances
  The dataset contains a label representing the hypernym of the given definition.
  ```json
  {
    'label': 'bonds',
    'definition': 'callable convertible bond is a kind of callable bond, convertible bond.'
  }
  ```
  
  #### Data Fields
  **label**: Can be one of the 17 predefined hypernyms.
  
  **definition**: Financial term definition relating to a concept or object in the financial domain.
  
  #### Data Splits
  The data contains training data with **317101** entries.
  
  #### Test set metrics
  The representational learning model is evaluated on a representative test set with 20% of the entries. The test set is scored based on the following metrics:
  * Average Accuracy
  * Mean Rank (position of the correct label in a set of 5 model predictions)
  
  We evaluate FinISH according to these metrics, where it outperforms other state-of-the-art sentence embeddings methods in this task.
  * Average Accuracy: **0.73**
  * Mean Rank: **1.61**
  
  ## Usage (Sentence-Transformers)
  Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:
  ```
  git clone https://github.com/huggingface/transformers.git
  pip install -q ./transformers
  pip install -U sentence-transformers
  ```
  Then you can use the model like this:
  ```python
  from sentence_transformers import SentenceTransformer, util
  import torch
  model = SentenceTransformer('yseop/roberta-base-finance-hypernym-identification')
  # Our corpus containing the list of hypernym labels
  hypernyms = ['Bonds',
  \t\t\t'Forward',
  \t\t\t'Funds',
  \t\t\t'Future',
  \t\t\t'MMIs',
  \t\t\t'Option',
  \t\t\t'Stocks',
  \t\t\t'Swap',
  \t\t\t'Equity Index',
  \t\t\t'Credit Index',
  \t\t\t'Securities restrictions',
  \t\t\t'Parametric schedules',
  \t\t\t'Debt pricing and yields',
  \t\t\t'Credit Events',
  \t\t\t'Stock Corporation',
  \t\t\t'Central Securities Depository',
  \t\t\t'Regulatory Agency']
  hypernym_embeddings = model.encode(hypernyms, convert_to_tensor=True)
  # Query sentences are financial terms to match to the predefined labels
  queries = ['Convertible bond', 'weighted average coupon', 'Restriction 144-A']
  # Find the closest 5 hypernyms of the corpus for each query sentence based on cosine similarity
  top_k = min(5, len(hypernyms))
  for query in queries:
      query_embedding = model.encode(query, convert_to_tensor=True)
      # We use cosine-similarity and torch.topk to find the highest 5 scores
      cos_scores = util.pytorch_cos_sim(query_embedding, hypern