title: Stochastic Projective Splitting: Solving Saddle-Point Problems with Multiple Regularizers
abstract: We present a new, stochastic variant of the projective splitting (PS) family of algorithms for monotone inclusion problems.  It can solve min-max and noncooperative game formulations arising in applications such as robust ML without the convergence issues associated with gradient descent-ascent, the current de facto standard approach in ML applications.  Our proposal is the first version of PS able to use stochastic gradient oracles. It can solve min-max games while handling multiple constraints and nonsmooth regularizers via projection and proximal operators. Unlike other stochastic splitting methods that can solve such problems, our method does not rely on a product-space reformulation of the original problem. We prove almost-sure convergence of the iterates to the solution and a convergence rate for the expected residual.  By working with monotone inclusions rather than variational inequalities, our analysis avoids the drawbacks of measuring convergence through the restricted gap function. We close with numerical experiments on a distributionally robust sparse logistic regression problem.
authors: Patrick R. Johnstone Jonathan Eckstein Thomas Flynn Shinjae Yoo
hIndex: Jonathan Eckstein 28