title: Lagrange multipler BFGS optimization with python
body: <p>I would like to use the scipy optimization routines, in order to minimize functions while applying some constraints. I would like to apply the Lagrange multiplier method, but I think that I missed something.</p>
  
  <p>My simple example: minimize f(x,y)=x^2+y^2, while keeping the constraint: y=x+4.0</p>
  
  <pre><code>import numpy as np
  from scipy.optimize import fmin_bfgs
  #X=[x,y,l]
  
  def f(X):
     x=X[0]
     y=X[1]
     return x**2+y**2
  
  def g(X):
     x=X[0]
     y=X[1]
     return y-x-4.000
  
  def L(X):
     l=X[2]
     return f(X)+l*g(X)
  
  
  def dL(X):
     x=X[0]
     y=X[1]
     l=X[2]
     gx=2.0*x
     gy=2.0*y
     gl=g(X)
     tmp=np.array([gx,gy,gl])
     return tmp
  
  
  x0=np.array([-2.0,2.0,0.0])
  
  
  
  print "f(x0)\t\t g(x0) \t\t L(x0)"
  print "%12.8f\t%12.8f\t%12.8f\t"%(f(x0),g(x0),L(x0))
  print "dL(x0)"
  print dL(x0)
  
  
  xopt=fmin_bfgs(L,x0,fprime=dL,disp=True)
  print xopt
  </code></pre>
  
  <p>Even if my x0 is on the spot, the optimization diverges, badly. Could someone please explain me how one should include the Lagrange multiplier properly and how one should initialize the multiplier?</p>