0.1 is a high learning rate, especially if you are using momentum. I looked at how googlenet (inception v1) was trained 1 and they used SGD with initial lr 0.01 and momentum 0.9 (quick_solver.prototxt also contains the details of their learning rate decay schedule).