Machine Learning, Optimization, Differentiation, Neural Networks, Control.
Forward mode (a.k.a Tangent mode) and Reverse mode (a.k.a Adjoint/Co-tangent mode), where the former processes the mathematical expression from input to output and latter from output to input. The working principle of SD is basically the same as the working principle of Forward mode AD, however the