Stochastic Gradient Descent


## SGD
**Vanilla Gradient Descent** without momentum
Alternative to [Batch gradient Descent](https://texonom.com/batch-gradient-descent-b25f0157eea74176bec7398fccb73dd5) 
Update more frequently than Batch Gradient Descent
model's parameters are updated after processing each training example
Very scalable so used in most model
update every time

### Need to shuffle for each [Epoch](https://texonom.com/epoch-57be10a7c12c43c799e96bac9962e7b8) 
### **Stochastic ****[Variational Inference](https://texonom.com/variational-inference-05563e34ddec467c95c9bda931770d50)**** **
KLD를 줄이는 쪽으로 파라메터를 업데이트
KLD 식이 미분 가능해야


> [변분추론(Variational Inference) · ratsgo's blog](https://ratsgo.github.io/generative%20model/2017/12/19/vi/)





