Lagrange Duality


We can make new [Dual optimization problem](https://texonom.com/dual-optimization-problem-1ac4c7beafc845e481c7f9eeaff666f9) using [Lagrange multiplier](https://texonom.com/lagrange-multiplier-1e414a17f1074b9eb4e5668ceacb46d6)
Optimal $d^*$ is dual optimization solution (optimal value of the objective)

Optimal value of Langrange $\theta$
Optimal of primal is $\theta_P$
Optimal of dual is $\theta_D$ 
$$\theta_P(w) = max_{\alpha, \beta}L(w, \alpha, \beta)\newline\theta_D(w) = min_wL(w, \alpha, \beta) \newline\theta_P(w) = f(w) \newline\theta_P(w) = \infty$$
infinity when w doesn’t satisfy constraints
목적함수 최대화하는 이유는 upper bound 찾기 위해, 상한선은 Primal problem에서 목적 함수를 최소화하는 데 도움줌

### Weak Duality Theorem
$$d^* = max_{\alpha, \beta}\; min_w \; L(w, \alpha, \beta) \leq min_w \;max_{\alpha, \beta}\; L(w, \alpha, \beta) = p^*$$
먼저 $\alpha, \beta$ 최대화해서 더 크다

알파의 최적값이 0보다 크다는 말은 g가 active하다는 말이고






