Probabilistic Line Searches for Stochastic Optimization
In deterministic optimization, line searches are a standard tool ensuring stability and efficiency. Where only stochastic gradients are available, no direct equivalent has so far been formulated, because uncertain gradients do not allow for a strict sequence of decisions collapsing the search space. We construct a probabilistic line search by combining the structure of existing deterministic methods with notions from Bayesian optimization. Our method retains a Gaussian process surrogate of the univariate optimization objective, and uses a probabilistic belief over the Wolfe conditions to monitor the descent. The algorithm has very low computational cost, and no user-controlled parameters. Experiments show that it effectively removes the need to define a learning rate for stochastic gradient descent.
One of the challenges of stochastic optimization with mini-batch gradient descent on, e.g., deep neural networks, is the selection of an appropriatelearning rate.In deterministic optimization, this choice is often avoided by using line search.This paper proposes a way to model the noise of stochastic objectives when performing line search and use Bayesian optimization to intelligently perform that search.It shows that careful choices for the covariance function can lead to efficient searches and that standard Wolfe conditions can be framed probabilistically.The paper provides recommendations for various meta-parameters in the algorithm.Empirical evaluation is performed on two-layer neural networks and show that the optimization is robust to initial learning rate choices.I love the idea of this paper.I've hoped someone would write this paper for a while, as it has become clear that Bayesian optimization should have something to say about traditional line search.I commend the authors for their thoughtful consideration of a variety of practical issues, from covariance choice, to the subtleties of weak vs strong Wolfe conditions, as well as their willingness to make practical recommendations about meta-parameters.My major complaint about the paper is that it does not follow through and examine whether this additional work provides a win over the increasingly-large literature on adapting learning rates.I really want this to be the "right" way to solve these kinds of problems, but I did not come away from the paper feeling like I should definitely use this.That is, here are some questions that I wanted to see empirically answered, but were not: - How sensitive is the method to variations in c_1, c_2, c_w, and \tau?- How does this method compare to algorithms that have scale adaptation such as  those in in [8], [9], [10], and [14]?- The choice of mini-batch size (10) is unusually small, and yet this has a  large impact on the noise.What is the impact of this choice?Other Comments and Questions: - The Wolfe conditions are a particular heuristic for balancing improvement in  the line search against starting a new one, given limited information about  the function.When exploring new probabilistic ideas, is it really necessary  to take this heuristic that seriously as a gold standard?- When the estimators y_t and y'_t are being computed, or when they are  computed for different values of t in the line search, is the same subset of  the data re-used?This was not clear to me, but the independence assumption  stated in Eq. (3) makes me think that they are different subsets.- What does it mean for an objective to be "irregular"?- I spent some time drawing samples from the covariance function in Eq. (4) and  did not find myself thinking these were interesting hypotheses for  objectives.They tended to be straight lines with small perturbations. Also,  I found that \tau had a large impact; one expects that for large values of  \tau, the min functions would essentially just return \tau.Thus \tau seems  to impact the effective scale of the problem.I wanted a more thoughtful  discussion of this issue.- Given that the paper is motivated by issues with line search for stochastic  objectives, a well-referenced and evidence-supported discussion of the  "brittleness" discussed in the introduction would've been helpful.Moreover,  it seems two things are ultimately conflated in the paper: 1) using Bayesian  optimization for line search in stochastic _or_ deterministic problem, and 2)  Bayesian optimization as a way to address the brittleness of line search in  stochastic optimization.- I don't really buy the need for analytic identification of the minima of the  mean function because the whole point of the Wolfe conditions is to avoid  the need to find exact minima; various heuristics for finding the minima  using, e.g., a grid, would be fine given the other costs involved.Also, it  is somewhat worrying to make potentially mismatched modeling decisions simply  to guarantee that it is easy to find the minimum between the evaluations.- I like the story told by Figure 2.- The explanation of c_w=0.3 was potentially interesting and should go into the  supplementary material.I suspect the authors have a careful argument to  make here that instead comes across as hand-wavy.Also, please evaluate the  empirical sensitivity to this choice.Presentational Issues:- P1 L32: "... if their gradient is corrupted ..." to "... if their gradients  are corrupted ..."- P1 L34: What is an "exchangeable" loss function?- P1 L36-40: The notation where j=1..m and also "indices j are i.i.d. draws"  doesn't really make sense and could perhaps be made more clear.- P1 L40: "... the error ... is unbiased ..." should probably be some more like  "... the estimator \hat{\mathcal{L}} of \mathcal{L} is unbiased ..."- P2 Figure 1: I found this figure somewhat confusing.I suggest drawing the  "true" function using a light color to make the decisions clearer.It would  be helpful if the value of c_1 was mentioned in the caption.I am not sure  that I understand the representation of the Wolfe conditions; is the idea  that if the tangent lines go through the orange regions then that point is  not satisfactory?- P3 L146-153: The notation y_i was previously used to denote the data in  Section 1 and it is here being reused to indicate noisy function evaluations.- P3 L133-153: The notation f is introduced as the objective, but the objective  was previously introduced as \mathcal{L} with the \hat{\mathcal{L}} being the  noisy estimate now represented as y.I would like there to be greater  precision in the notation describing these functions.- P3 Figure 2: The top figure is very busy for how small it is.I suggest  removing the sample paths, i.e., the dashed lines.- P4 L169-192: The somewhat unusual notation for partial derivatives would  benefit from a one-sentence explanation.- P5 L231: I am assuming that \eta is the minimum of the mean over actual  previous evaluations and not candidates, but a sentence that explains this  would be helpful.- P5 L258: The probability of Wolfe conditions being satisfied is a _truncated_  bivariate normal probability, I think.- P5 Figure 3: Very little explanation of this figure in the caption and it  doesn't seem to be referred to in the text at all.- P8 Figure 4: This figure is not well explained.What do the colors mean  relative to the top and the bottom?Are the colors on the bottom meant to  refer to the different vertical columns, i.e., initial learning rates of the  top?