i came across this problem yesterday. a quick fix seems to be lowering your learning rate. I tried it with 1e-2, it started throwing nans, nans,nans.
I made it to 1e-3. converged fine.
this 306 might be helpful, otherwise.