Could you just for the sake of debugging set the learning rate quite high, e.g. 100 and run a single update step to check if the parameters get updated?
I would like to make sure that the gradients are not too small and we are thus not seeing any updates in the parameters even though the code should generally work.
If that still doesn’t work, could you point me to some lines of code in your Gist so that debugging would be a bit faster?