Body: <p>In section 6.5.6 of the book <a href="https://www.deeplearningbook.org" rel="nofollow noreferrer">Deep Learning</a> by Ian et. al. general backpropagation algorithm is described as:</p>
  <blockquote>
  <p>The back-propagation algorithm is very simple. To compute the gradient of some
  scalar z with respect to one of its ancestors x in the graph, we begin by observing
  that the gradient with respect to z is given by dz = 1. We can then compute dz
  the gradient with respect to each parent of z in the graph by multiplying the current gradient by the Jacobian of the operation that produced z. We continue multiplying by Jacobians traveling backwards through the graph in this way until we reach x. For any node that may be reached by going backwards from z through two or more paths, we simply sum the gradients arriving from different paths at that node.</p>
  </blockquote>
  <p>To be specific I don't get this part:</p>
  <blockquote>
  <p>We can then compute dz the gradient with respect to each parent of z in the graph by multiplying the current gradient by the Jacobian of the operation that produced z.</p>
  </blockquote>
  <p>Can anyone help me understand this with some illustration? Thank you.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-29T05:05:59.497Z
Comments.Text: Hey. 3 blue 1 brown made amazing series on neural networks, https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi, the 3rd and 4th part have good illustrations of back propagation algorithm
Comments.UserDisplayName: mark mark
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-29T11:51:41.063Z
Comments.Text: Please, ask a more specific question. The current question is too broad.
Comments.UserDisplayName: nbro
Tags: <deep-learning><backpropagation>
Title: I need help understanding general back propagation algorithm
UserDisplayName: Bhuwan Bhatt
ThreadText: # I need help understanding general back propagation algorithm
  
  (Asked by: username_0 on 2020-06-29)
  
  In section 6.5.6 of the book [Deep Learning](https://www.deeplearningbook.org) by Ian et. al. general backpropagation algorithm is described as:
  
  > The back-propagation algorithm is very simple. To compute the gradient of some
  > scalar z with respect to one of its ancestors x in the graph, we begin by observing
  > that the gradient with respect to z is given by dz = 1. We can then compute dz
  > the gradient with respect to each parent of z in the graph by multiplying the current gradient by the Jacobian of the operation that produced z. We continue multiplying by Jacobians traveling backwards through the graph in this way until we reach x. For any node that may be reached by going backwards from z through two or more paths, we simply sum the gradients arriving from different paths at that node.
  
  To be specific I don't get this part:
  
  > We can then compute dz the gradient with respect to each parent of z in the graph by multiplying the current gradient by the Jacobian of the operation that produced z.
  
  Can anyone help me understand this with some illustration? Thank you.
  
  **Tags:** deep-learning, backpropagation
  > **Comments:**
  > **username_1**: Hey. 3 blue 1 brown made amazing series on neural networks, https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi, the 3rd and 4th part have good illustrations of back propagation algorithm  
  > **username_2**: Please, ask a more specific question. The current question is too broad.  
  
  ---
  
  ## 0 answers
  
  