Hi everyone,
I start using pytorch today. However, I find it’s too slow in the skip-gram model I implement. I compare it with the same program implemented with DyNet. The result shows DyNet is 1000 times faster. I don’t believe that!
So I want to ask if anyone has implemented skip-gram model that is as fast as the existing tools (e.g. word2vec).
Any suggestions are welcomed!
=============SOLUTION===============
I solved this problem. It’s caused by the using of param momentum, which will cause the dense update of the optimizer (SGD), instead of the sparse update.
The code is here 162