Linear Regression
[Objective function](https://texonom.com/objective-function-5bf6c1fc7c854b5aaaacd17d5e094464) 


### Linear Predictor (minimal [Perceptron](https://texonom.com/perceptron-1deb66f486d54c93bb928d8afba1864c))
$$h(x) = \Sigma_{i=0}^{d}\theta_ix_i = \theta^Tx$$
in Linear Regression there is always global optimum
- update $\theta$ - [Gradient Descent](https://texonom.com/gradient-descent-c1342b13182f4fb6959023a75b5e2ff8)  like algorithms
> - add gradient of derivative of cost function J
$$J(\theta) = \frac{1}{2}\Sigma_{i=1}^n(h_\theta(x^{(i)}) - y^{(i)})^2 = \frac{1}{2}(X\theta - \vec{y})^T(X\theta - \vec{y})$$
$$\theta := \theta + \alpha \Sigma(y_i - h_\theta(x_i))x_i$$
- Closed form - solve equation - assume $X^TX$ is invertible
$$\theta = (X^TX)^{-1}X^T\vec{y}$$
### Linear Regression Notion
|Title|
|:-:|
|[Basis Function](https://texonom.com/basis-function-994de24f84dd437d9d563ca2a87652d6)|
|[Locally weighted Linear Regression](https://texonom.com/locally-weighted-linear-regression-6f405fcc12f64e0b9ec061c87d81e1e2)|
|[Ridge regression](https://texonom.com/ridge-regression-8520dc9911234f62bc6fa25976af0972)|
|[Lasso Regression](https://texonom.com/lassoregression-01bff5049e0e40faa3c1ce3fb68f1c76)|
|[LMS](https://texonom.com/lms-3646b634f8974a51b8184a23f3defa1d)|







> [선형회귀 - 공돌이의 수학정리노트](https://angeloyeo.github.io/2020/08/24/linear_regression.html)





