Hi, How could I use genism dicitionary with torchtext.
where docs is dataset after cleaning like [“cat” “sit” “table”]
docs_dict = Dictionary(docs)
docs_dict.filter_extremes(no_below=20, no_above=0.2)
docs_dict.compactify()
docs_corpus = [docs_dict.doc2bow(doc) for doc in docs]
model_tfidf = TfidfModel(docs_corpus, id2word=docs_dict)
docs_tfidf  = model_tfidf[docs_corpus]
docs_tfid   = np.vstack([sparse2full(c, len(docs_dict)) for c in docs_tfidf])
golve_vecs = np.vstack([nlp(docs_dict[i]).vector for i in range(len(docs_dict))])
docs_embedded_with_tfid = np.dot(docs_tfid,golve_vecs)
where nlp is glove embedding
any help or healthy discussion please.
Thanks.