Data Science, Machine Learning, NLP, Word Embeddings, AI.
similarity issue. Ideally we would want similar words like “cat” and “tiger” to have somewhat similar features. But with these one-hot vectors, “cat” is as similar to “tiger” as literally any other word, which isn’t great. A related point is that we might want to do analogy-like vector operations