If accuracy is important to you, SGD is certainly a safe bet. However, it is also incredibly slow.
But nothing in “the book” says you can’t start with a faster optim, such as Adagrad or Adadelta, and then switch to SGD for the final few epochs, just to squeeze that last bit of accuracy out of a model. I find this to be the best approach.
Happy inferencing,