Machine Learning, Deep Learning, Theano, TensorFlow, Framework.
nonlinearities like ReLU (Rectified Linear Units) are not differentiable at some points. So, we instead calculate the gradient in an iterative manner. Since the second method could be used universally, most computational graph packages like Computation Graph Toolkit(http://rll.berkeley.edu/cgt/)