I don’t know, if this was the issue, as the author of the post never followed up.
Note that tanh would have a similar behavior of the gradient and of course your output would be in the range [-1, 1], but you could try it out for your model.