<p>Yes, your interpretation is correct.</p>
<p>This trick reflects that there are many possible character n-grams, but only relatively few of them contribute to the word's meaning. Instead of resolving what n-grams are the most salient ones, they just opt for a stochastic solution.</p>

CC BY-SA 4.0
2023-05-04T07:53:35.227Z
2023-05-04T07:53:35.227Z
Jindřich
<p>As per Section 3.2 in the <a href="https://arxiv.org/pdf/1607.04606.pdf" rel="nofollow noreferrer">original paper on Fasttext</a>, the authors state:</p>
<blockquote>
<p>In order to bound the memory requirements of our model, we use a
hashing function that maps n-grams to integers in 1 to K</p>
</blockquote>
<p>Does this mean the model computes only K embeddings regardless of the number of distinct ngrams extracted from the training corpus, and if 2 different ngrams collide when hashed, they share the same embedding?</p>
<p>Thanks.</p>

<neural-networks><natural-language-processing><word-embedding><text-classification><pretrained-models>
Do different ngrams share embedding in Fasttext?
Fijoy Vadakkumpadan
# Do different ngrams share embedding in Fasttext?

(Asked by: username_0 on 2023-04-20)

As per Section 3.2 in the [original paper on Fasttext](https://arxiv.org/pdf/1607.04606.pdf), the authors state:

> In order to bound the memory requirements of our model, we use a
> hashing function that maps n-grams to integers in 1 to K

Does this mean the model computes only K embeddings regardless of the number of distinct ngrams extracted from the training corpus, and if 2 different ngrams collide when hashed, they share the same embedding?

Thanks.

**Tags:** neural-networks, natural-language-processing, word-embedding, text-classification, pretrained-models


---

## 1 answers

### Answer by username_1 on 2023-05-04. Score: 1 | Accepted answer

Yes, your interpretation is correct.

This trick reflects that there are many possible character n-grams, but only relatively few of them contribute to the word's meaning. Instead of resolving what n-grams are the most salient ones, they just opt for a stochastic solution.


---

