Answers.Body: <p>If by "symbolic" you mean finding an analytical solution, that is, an equation for each weight, then the answer is no. The example you chose results in a system linear equations, which can be solved analytically. However once you introduce non linearities (by using activation functions with more than one layer), most non trivial cases will have no analytical solution and will need to be solved numerically. This is not a problem specific to tensorflow, it is  a mathematical issue, it will not be possible on any language, current or future. Unless there is some revolution in math first.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-10T09:31:36.480Z
Answers.LastActivityDate: 2018-05-10T09:31:36.480Z
Answers.UserDisplayName: Wolphram jonny
Body: <p>From <a href="https://stackoverflow.com/questions/36370129/does-tensorflow-use-automatic-or-symbolic-gradients">https://stackoverflow.com/questions/36370129/does-tensorflow-use-automatic-or-symbolic-gradients</a>, I understood TensorFlow requires all the operations in the Graph to be explicit formulas (instead of black-boxes, such as raw python functions) to do Automatic Differentiation. Then it will do some kind of Gradient Descent based on that to minimization.</p>
  
  <p>I'm wondering, since it already know all the explicit formulas, can it directly find out the minimum by examining the equation itself? Like computing the points where gradient is zero or do not exist, then do some kind of processing to find out the minimum.</p>
  
  <p>I found it is simple to do this "symbolic minimization" above with few variables such as minimizing <code>Σ(a_i - v)^2</code> where <code>v</code> is the trainable variable an <code>a_i</code> are all the training samples. I'm not sure is there a general way though.</p>
  
Tags: <tensorflow><optimization>
Title: Can TensorFlow minimize "symbolically"
UserDisplayName: ZisIsNotZis
ThreadText: # Can TensorFlow minimize "symbolically"
  
  (Asked by: username_0 on 2018-05-10)
  
  From <https://stackoverflow.com/questions/36370129/does-tensorflow-use-automatic-or-symbolic-gradients>, I understood TensorFlow requires all the operations in the Graph to be explicit formulas (instead of black-boxes, such as raw python functions) to do Automatic Differentiation. Then it will do some kind of Gradient Descent based on that to minimization.
  
  I'm wondering, since it already know all the explicit formulas, can it directly find out the minimum by examining the equation itself? Like computing the points where gradient is zero or do not exist, then do some kind of processing to find out the minimum.
  
  I found it is simple to do this "symbolic minimization" above with few variables such as minimizing `Σ(a_i - v)^2` where `v` is the trainable variable an `a_i` are all the training samples. I'm not sure is there a general way though.
  
  **Tags:** tensorflow, optimization
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-05-10. Score: 0 | Accepted answer
  
  If by "symbolic" you mean finding an analytical solution, that is, an equation for each weight, then the answer is no. The example you chose results in a system linear equations, which can be solved analytically. However once you introduce non linearities (by using activation functions with more than one layer), most non trivial cases will have no analytical solution and will need to be solved numerically. This is not a problem specific to tensorflow, it is a mathematical issue, it will not be possible on any language, current or future. Unless there is some revolution in math first.
  
  
  ---
  
  