readme: ---
  language:
  - en
  library_name: fasttext
  pipeline_tag: text-classification
  tags:
  - text
  - semantic-similarity
  - earnings-call-transcripts
  - word2vec
  - fasttext
  widget:
  - text: "transformation"
    example_title: "transformation"
  - text: "sustainability"
    example_title: "sustainability"
  - text: "turnaround"
    example_title: "turnaround"
  - text: "disruption"
    example_title: "disruption"
  ---
  
  # EarningsCall2Vec
  
  **EarningsCall2Vec** is a [`fastText`](https://fasttext.cc/) word embedding model that was trained via [`Gensim`](https://radimrehurek.com/gensim/). It maps each token in the vocabulary to a dense, 300-dimensional vector space, designed for performing **semantic search**. More details about the training procedure can be found [below](#model-training).
  
  
  ## Background
  
  Context on the project. 
  
  
  ## Usage
  
  The model is intented to be used for semantic search: It encodes the search-query in a dense vector space and finds semantic neighbours, i.e., token which frequently occur within similar contexts in the underlying training data. The query should consist of a single word. When provided a bi-, tri-, or even fourgram, the quality of the model output depends on the presence of the query token in the model's vocabulary. Multiple words are to be concated by an underscore (e.g., "machine_learning" or "artifical_intelligence").
  
  
  ## Usage (API)
  
  ```python
  import json
  import requests
  
  API_TOKEN = <TOKEN>
  
  headers = {"Authorization": f"Bearer {API_TOKEN}"}
  API_URL = "https://api-inference.huggingface.co/models/simonschoe/call2vec"
  
  def query(payload):
      data = json.dumps(payload)
      response = requests.request("POST", API_URL, headers=headers, data=data)
      return json.loads(response.content.decode("utf-8"))
  
  query({"inputs": "<insert-query-here"})
  ```
  
  ## Usage (Gensim)
  
  ```python
  from huggingface_hub import hf_hub_url, cached_download
  from gensim.models.fasttext import load_facebook_model
  
  # download model from huggingface hub
  url = hf_hub_url(repo_id="simonschoe/call2vec", filename="model.bin")
  cached_download(url)
  
  # load model via gensim
  model = load_facebook_model(<PATH-MODEL>)
  
  # extract word embeddings
  model.wv['transformation']
  # get similar phrases
  model.wv.most_similar(positive='transformation', topn=5)
  # get dissimilar phrases
  model.wv.most_similar(negative='transformation', topn=5, restrict_vocab=None)
  # compute pairwise similarity scores (distance = 1 - similarity)
  model.wv.similarity('transformation', 'continuity')
  ```
  
  If model size is crucial, the final model could be additionally compressed using the [`compress-fasttext`](https://github.com/avidale/compress-fasttext) library (e.g., via pruning, conversion to `float16`, or product quantization).
  
  
  ## Model Training
  
  The model has been trained on text data stemming from earnings call transcripts. The data is restricted to a call's question-and-answer (Q&A) section and the remarks by firm executives. The data has been preprocessed prior to model training via stop word removal, lemmatization, named entity masking, and coocurrence modeling.
  
  **Corpus statistics:**
  - Total number of calls: 159973
  - Total number of remarks: 4512535
  - Total number of token: 219228853
  
  The following code snippets presents the training pipeline:
  ```python
  import logging
  from gensim.models import FastText
  from gensim.models.word2vec import LineSentence
  from gensim.models.callbacks import CallbackAny2Vec
  from gensim.models.fasttext import save_facebook_model
  
  # init
  model = FastText(
      vector_size=300,
      window=5,
      min_count=10,
      alpha=0.025,
      negative = 5,
      seed=2021,
      sample = 0.001,
      sg=1,
      hs=0,
      max_vocab_size=None,
      workers=10,
  )
  
  # build vocab
  model.build_vocab(corpus_iterable=LineSentence(<PATH-TRAIN-DATA>))
  
  # train model
  logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO)
  
  class MyCallback(CallbackAny2Vec):
      def __init__(self):
          self.epoch = 0
          
      def on_epoch_end(self, model):
          self.epoch += 1
          if (self.epoch % 10) == 0:
              # save in gensim format
              model.save(<PATH-SAVE>)
          
      def on_train_end(self, model):
          # save in binary format for upload to huggingface
          save_facebook_model(<PATH-SAVE>.bin)
      
  model.train(
      corpus_iterable=LineSentence(<PATH-TRAIN-DATA>),
      total_words=model.corpus_total_words,
      total_examples=model.corpus_count,
      epochs=<EPOCHS>,
      callbacks=[MyCallback()],
  )
  ```
  
  **Model statistics:**
  - Vocabulary size: 64,891
  - Min. token frequency: 10
  - Embedding dimensions: 300
  - Number of epochs: 50
  