You are correct about the momentum option.
Upon looking at the code, I realized that the problem is actually elsewhere. You should call optimizer.zero_grad() at beginning of each loop  Otherwise, the gradient will just accumulate each loop (so its actually optimizing the wrong gradient values), and without calling coalesce(), sparse gradient can grow very large (each x index will have multiple values), causing the slow down.
After adding the zero_grad call, on current master, sparse version gives me 3.0683s and dense version gives me 7.5844s.
About updating binary. we will do so once we release a new version. Currently, if you need the new features/fixes, you can build from source by yourself. It’s very very simple to do so.