Machine Learning, Optimization, Differentiation, Neural Networks, Control.
optimization problem. Typically, the derivative based methods could be classified into two types which are the first order methods and second order methods. The first order methods primarily requires two information to solve the underlying nonlinear optimization problem which are Objective and