Adaptive Stochastic Optimization: From Sets to Paths
Adaptive stochastic optimization (ASO) optimizes an objective function adaptively under uncertainty. It plays a crucial role in planning and learning under uncertainty, but is, unfortunately, computationally intractable in general. This paper introduces two conditions on the objective function, the marginal likelihood rate bound and the marginal likelihood bound, which, together with pointwise submodularity, enable efficient approximate solution of ASO. Several interesting classes of functions satisfy these conditions naturally, e.g., the version space reduction function for hypothesis learning. We describe Recursive Adaptive Coverage, a new ASO algorithm that exploits these conditions, and apply the algorithm to two robot planning tasks under uncertainty. In contrast to the earlier submodular optimization approach, our algorithm applies to ASO over both sets and paths.
*Summary*This paper studies the adaptive stochastic optimization on paths, by imposing some constraints (namely the MLRB and the MLB conditions) on the relationship between the probability of a realization and its utility. While the notion of *adaptive submodularity* states that in expectation, no item will "surprise" us given a longer history, the MLRB / MLB conditions in this paper roughly characterizes the fact that such "surprise" only happens with small probability (i.e., realization with small probability has large utility). By leveraging such constraints, the authors propose a novel recursive algorithm with near-optimal guarantee on the expected cost needed to reach a certain coverage. Under the MLB conditions, the paper recovers the classical approximation result on bayesian pool-based active learning, up to a constant factor.*Quality*The paper is nicely written and well structured. Regarding the experimental results: by Theorem 4, it would be very interesting to see how the proposed RAC algorithm competes with existing algorithms *empirically* on the pool-based active learning problem (e.g., [3][5][7], or [6] under the bounded noise setting), given the fact that the version space reduction function and the Gibbs error function both satisfy MLRB condition, pointwise submodularity condition, and adaptive submodularity condition. Having these results will certainly make the paper stronger.*Clarity*Most of the paper is clear. It would be helpful if the authors elaborate more on the intuitive explanations of the two conditions: why are those two conditions more natural than the adaptive submodularity condition for adaptive optimization problems on paths?*Originality and Significance*To the best of my knowledge, the technical contribution of this contribution is novel. It provides interesting insights to developing tractable algorithms for the adaptive stochastic optimization problem.