The learning rates don’t mean anything to me, and 3e-4 or so isn’t uncommon either. But if you can deal with coordinates outside [-1,1], one option would be to use 1.5*tanh - you’d bound yourself to a reasonable range while not saturating.
Jeremy Howard’s fast.ai does something like that for predicting movielens ratings in his collaborative filtering lecture (or at least has done that in some version of the course).
Best regards
Thomas