I implemented a word2vec (skip-gram with ns) using pytorch, but it’s running much much slower than the gensim version of word2vec.
gensim took like 2min to finish the training, whereas the pytorch version seems will take half a day though.
The setup of pytorch is on Macbook pro with only CPU, and I can see the CPU of the pytorch word2vec is 100% (should be only 1 core utilized).
Whereas gensim uses multiple processes to run the training, it’s using numpy and cython functions (release GIL).
So I wonder how could I optimize the pytorch implementation further to match the performance of gensim version?