The Moreau envelope (or the Moreau-Yosida regularization) formula_1 of a proper lower semi-continuous convex function formula_2 is a smoothed version of formula_2. It was proposed by Jean-Jacques Moreau in 1965.
The Moreau envelope has important applications in mathematical optimization: minimizing over formula_1 and minimizing over formula_2 are equivalent problems in the sense that the sets of minimizers of formula_2 and formula_1 are the same. However, first-order optimization algorithms can be directly applied to formula_1, since formula_2 may be non-differentiable while formula_1 is always continuously differentiable. Indeed, many proximal gradient methods can be interpreted as a gradient descent method over formula_1.
Definition.
The Moreau envelope of a proper lower semi-continuous convex function formula_2 from a Hilbert space formula_13 to formula_14 is defined as
formula_15
Given a parameter formula_16, the Moreau envelope of formula_17 is also called as the Moreau envelope of formula_2 with parameter formula_19.
Properties.
formula_22. By defining the sequence formula_23 and using the above identity, we can interpret the proximal operator as a gradient descent algorithm over the Moreau envelope.
formula_24 where formula_25 denotes the convex conjugate of formula_2.
Since the subdifferential of a proper, convex, lower semicontinuous function on a Hilbert space is inverse to the subdifferential of its convex conjugate, we can conclude that if formula_27 is the maximizer of the above expression, then formula_28 is the minimizer in the primal formulation and vice versa.
References.
 ;templatestyles src="Reflist/styles.css" / ;