Moreau envelope
The Moreau envelope (or the Moreau-Yosida regularization)  of a proper lower semi-continuous convex function  is a smoothed version of . It was proposed by Jean-Jacques Moreau in 1965. The Moreau envelope has important applications in mathematical optimization: minimizing over  and minimizing over  are equivalent problems in the sense that set of minimizers of   and  are the same. However, first-order optimization algorithms can be directly applied to , since  may be non-differentiable while  is always continuously differentiable. Indeed, many proximal gradient methods can be interpreted as a gradient descent method over . Definition 
The Moreau envelope of a proper lower semi-continuous convex function  from a Hilbert space  to  is defined as

Given a parameter , the Moreau envelope of  is also called as the Moreau envelope of  with parameter . Properties 

 The Moreau envelope can also be seen as the infimal convolution between  and . The proximal operator of a function is related to the gradient of the Moreau envelope by the following identity:
. By defining the sequence  and using the above identity, we can interpret the proximal operator as a gradient descent algorithm over the Moreau envelope. Using Fenchel's duality theorem, one can derive the following dual formulation of the Moreau envelope: 
 where  denotes the convex conjugate of . Since the subdifferential of a proper, convex, lower semicontinuous function on a Hilbert space is inverse to the subdifferential of its convex conjugate, we can conclude that if  is the maximizer of the above expression, then  is the minimizer in the primal formulation and vice versa. By Hopf-Lax formula, the Moreau envelope is a viscosity solution to a Hamilton-Jacobi equation. Stanley Osher and co-authors used this property and Cole-Hopf transformation to derive an algorithm to compute approximations to the proximal operator of a function. See also 

 Proximal operator
 Proximal gradient method

References

External links 

 
 A Hamilton-Jacobi-based Proximal Operator: a YouTube video explaining an algorithm to approximate the proximal operator

Mathematical optimization