If the problem is local minima, decreasing learning rate won’t help i think (correct me if I’m wrong).
Also it would be strange if the problem would be local minima -the loss is not decreasing at all, so I would be v. unlucky if I started training with random weigths from local minima.
I don’t think this is the problem, but I’ll try messing with learning rate both dircetions.