Answers.Body: <p>Yes, your interpretation is correct.</p>
  <p>This trick reflects that there are many possible character n-grams, but only relatively few of them contribute to the word's meaning. Instead of resolving what n-grams are the most salient ones, they just opt for a stochastic solution.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-04T07:53:35.227Z
Answers.LastActivityDate: 2023-05-04T07:53:35.227Z
Answers.UserDisplayName: Jindřich
Body: <p>As per Section 3.2 in the <a href="https://arxiv.org/pdf/1607.04606.pdf" rel="nofollow noreferrer">original paper on Fasttext</a>, the authors state:</p>
  <blockquote>
  <p>In order to bound the memory requirements of our model, we use a
  hashing function that maps n-grams to integers in 1 to K</p>
  </blockquote>
  <p>Does this mean the model computes only K embeddings regardless of the number of distinct ngrams extracted from the training corpus, and if 2 different ngrams collide when hashed, they share the same embedding?</p>
  <p>Thanks.</p>
  
Tags: <neural-networks><natural-language-processing><word-embedding><text-classification><pretrained-models>
Title: Do different ngrams share embedding in Fasttext?
UserDisplayName: Fijoy Vadakkumpadan
ThreadText: # Do different ngrams share embedding in Fasttext?
  
  (Asked by: username_0 on 2023-04-20)
  
  As per Section 3.2 in the [original paper on Fasttext](https://arxiv.org/pdf/1607.04606.pdf), the authors state:
  
  > In order to bound the memory requirements of our model, we use a
  > hashing function that maps n-grams to integers in 1 to K
  
  Does this mean the model computes only K embeddings regardless of the number of distinct ngrams extracted from the training corpus, and if 2 different ngrams collide when hashed, they share the same embedding?
  
  Thanks.
  
  **Tags:** neural-networks, natural-language-processing, word-embedding, text-classification, pretrained-models
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-05-04. Score: 1 | Accepted answer
  
  Yes, your interpretation is correct.
  
  This trick reflects that there are many possible character n-grams, but only relatively few of them contribute to the word's meaning. Instead of resolving what n-grams are the most salient ones, they just opt for a stochastic solution.
  
  
  ---
  
  