Body: <p>I recently came across symbol-to-symbol and symbol-to-number differentiation, out of which symbol to symbol seemed fairly straightforward - the computational graph is extended to include gradient calculations and relationships between gradients.</p>
  
  <p>I have a problem in understanding what exactly symbol-to-number differentiation is. Does it map directly every variable in the backprop to its relevant gradient? If yes, how does it do this without knowing about the rest of the computational graph? </p>
  
  <p>If the question is unclear, to increase context - TensorFlow uses symbol to symbol differentiation whereas torch uses symbol to number (apparently). </p>
  
  <p>Came across this in section 6.5.5 of <a href="https://www.deeplearningbook.org/contents/mlp.html" rel="nofollow noreferrer">Deep Learning, the book</a>. The material mentioned has a convincing explanation for symbol-to-symbol differentiation but could not say the same for symbol-to-number differentiation. </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-08T08:09:29.797Z
Comments.Text: @DuttaA - Assuming you mean where I came across this - Section 6.5.5 of https://www.deeplearningbook.org/contents/mlp.html
Comments.UserDisplayName: ashenoy
Tags: <neural-networks><deep-learning><backpropagation><computational-complexity><symbolic-computing>
Title: What is symbol-to-number differentiation?
UserDisplayName: ashenoy
ThreadText: # What is symbol-to-number differentiation?
  
  (Asked by: username_0 on 2020-04-07)
  
  I recently came across symbol-to-symbol and symbol-to-number differentiation, out of which symbol to symbol seemed fairly straightforward - the computational graph is extended to include gradient calculations and relationships between gradients.
  
  I have a problem in understanding what exactly symbol-to-number differentiation is. Does it map directly every variable in the backprop to its relevant gradient? If yes, how does it do this without knowing about the rest of the computational graph?
  
  If the question is unclear, to increase context - TensorFlow uses symbol to symbol differentiation whereas torch uses symbol to number (apparently).
  
  Came across this in section 6.5.5 of [Deep Learning, the book](https://www.deeplearningbook.org/contents/mlp.html). The material mentioned has a convincing explanation for symbol-to-symbol differentiation but could not say the same for symbol-to-number differentiation.
  
  **Tags:** neural-networks, deep-learning, backpropagation, computational-complexity, symbolic-computing
  > **Comments:**
  > **username_0**: @DuttaA - Assuming you mean where I came across this - Section 6.5.5 of https://www.deeplearningbook.org/contents/mlp.html  
  
  ---
  
  ## 0 answers
  
  