How does the loss curve look like?
Is it decreasing at all or just rising?
In the latter case, could you try to lower the learning rate even more?
Also, what kind of data are you using?
What is the min, max, mean, std of your data?
Normalizing might help, if you are dealing with a large range of values.