et al., 2017). Instead of performing a simple lookup operation each word is hashed multiple times
and the hashes are modded into a vector table. Each word is then represented by a combination of
vectors. As such hash embeddings represent a large number of words using a much smaller number
of vectors. While spaCy models historically use standard static vectors, in the future we are planning
to release pretrained hash embeddings trained with our ﬂoret library5 that combines fastText with
hash embeddings.
In this report, we describe hash embeddings and compare their performance to that of standard
word embedding lookups. In addition, we fuse subword and word shape information into the word
representations rather than representing each orthographic form with a vector. This process helps
alleviate the issue of word forms being unseen during training and establish links between words that
are morphologically related. We perform this comparison across multiple named entity recognition
MULTI HASH EMBEDDINGS IN SPACY