Answers.Body: <p>'Backprop' is short for 'backpropagation of error' in order to avoid confusion when using <em>backpropagation</em> term.</p>
  
  <p>Basically <em>backpropagation</em> refers to the method for computing the gradient of the case-wise error function with respect to the weights for a feedforward network<sup>Werbos</sup>. And <em>backprop</em> refers to a training method that uses backpropagation to compute the gradient.</p>
  
  <p>So we can say that a <em>backprop</em> network is a feedforward network trained by <em>backpropagation</em>.</p>
  
  <p>The 'standard backprop' term is a euphemism for the <em>generalized delta rule</em> which is most widely used supervised training method.</p>
  
  <p>Source: <a href="ftp://ftp.sas.com/pub/neural/FAQ2.html#A_backprop" rel="noreferrer">What is backprop?</a> at FAQ of Usenet newsgroup comp.ai.neural-nets</p>
  
  <p>References:</p>
  
  <ul>
  <li>Werbos, P. J. (1974). Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. PhD thesis, Harvard University.</li>
  <li>Werbos, P. J. (1994). The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting,Wiley Interscience.</li>
  <li>Bertsekas, D. P. (1995), Nonlinear Programming, Belmont, MA: Athena Scientific, ISBN 1-886529-14-0.</li>
  <li>Bertsekas, D. P. and Tsitsiklis, J. N. (1996), Neuro-Dynamic Programming, Belmont, MA: Athena Scientific, ISBN 1-886529-10-8.</li>
  <li>Polyak, B.T. (1964), "Some methods of speeding up the convergence of iteration methods," Z. Vycisl. Mat. i Mat. Fiz., 4, 1-17.</li>
  <li>Polyak, B.T. (1987), Introduction to Optimization, NY: Optimization Software, Inc.</li>
  <li>Reed, R.D., and Marks, R.J, II (1999), Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks, Cambridge, MA: The MIT Press, ISBN 0-262-18190-8.</li>
  <li>Rumelhart, D.E., Hinton, G.E., and Williams, R.J. (1986), "Learning internal representations by error propagation", in Rumelhart, D.E. and McClelland, J. L., eds. (1986), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1, 318-362, Cambridge, MA: The MIT Press.</li>
  <li>Werbos, P.J. (1974/1994), The Roots of Backpropagation, NY: John Wiley &amp; Sons. Includes Werbos's 1974 Harvard Ph.D. thesis, Beyond Regression.</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-27T01:29:44.367Z
Answers.Text: maybe you need to accept your own answer.
Answers.UserDisplayName: rubengavidia0x
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-03T14:39:02.827Z
Answers.LastActivityDate: 2016-08-03T14:39:02.827Z
Answers.UserDisplayName: kenorb
Answers.Body: <p>Yes, as Franck has rightly put, "backprop" means backpropogation, which is frequently used in the domain of neural networks for error optimization.</p>
  
  <p>For a detailed explanation, I would point out <a href="http://neuralnetworksanddeeplearning.com/chap2.html" rel="nofollow">this tutorial</a> on the concept of backpropogation by a very good book of Michael Nielsen. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-02T16:54:40.380Z
Answers.LastActivityDate: 2016-08-02T16:54:40.380Z
Answers.UserDisplayName: Dawny33
Answers.Body: <p>It's a fancy name for the multivariable chain rule.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-23T21:06:23.717Z
Answers.Text: Hello.  Could you support this answer with a little more detail?  (I realize it's a simple question, but I think your answer is worthy of greater explication.)
Answers.UserDisplayName: DukeZhou
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-10T00:30:14.197Z
Answers.Text: @FourierFlux Please, follow the suggestion in the previous comment. At least, provide a link to an external resource, such as Wiki, that provides more details. Take a look at https://ai.stackexchange.com/help/how-to-answer to know more about what we expect from answers and how to write a good answer.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-22T00:49:46.747Z
Answers.LastActivityDate: 2020-04-22T00:49:46.747Z
Answers.UserDisplayName: FourierFlux
Answers.Body: <p>We need to compute the gradients in-order to train the deep neural networks. Deep neural network consists of many layers. Weight parameters are present between the layers. Since we need to compute the gradients of loss function for each weight, we use an algorithm called backprop. It is an abbreviation for <strong>backprop</strong>agation, which is also called as error backpropagation or reverse differentiation.</p>
  <p>It can be understood well from the following paragraph taken from <a href="https://web.stanford.edu/%7Ejurafsky/slp3/7.pdf" rel="nofollow noreferrer">Neural Networks and Neural Language Models</a></p>
  <blockquote>
  <p>For deep networks, computing the gradients for each weight is much
  more complex,since we are computing the derivative with respect to
  weight parameters that appear all the way back in the very early
  layers of the network, even though the loss is computed only at the
  very end of the network.<strong>The solution to computing this gradient is an
  algorithm called error backpropagation or backprop</strong>. While backprop was
  invented for neural networks, it turns out to be the same as a more
  general procedure called backward  differentiation, which depends on
  the  notion  of computation graphs.</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-08T10:32:25.017Z
Answers.LastActivityDate: 2021-07-08T10:45:23.250Z
Answers.LastEditDate: 2021-07-08T10:45:23.250Z
Answers.UserDisplayName: hanugm
Answers.Body: <p>"Backprop" is the same as "backpropagation": it's just a shorter way to say it. It is sometimes abbreviated as "BP".</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-02T15:40:24.820Z
Answers.LastActivityDate: 2016-08-02T15:40:24.820Z
Answers.UserDisplayName: Franck Dernoncourt
Body: <p>What does "backprop" mean? Is the "backprop" term basically the same as "backpropagation" or does it have a different meaning?</p>
  
Tags: <neural-networks><backpropagation><terminology><definitions>
Title: What is "backprop"?
UserDisplayName: kenorb
ThreadText: # What is "backprop"?
  
  (Asked by: username_0 on 2016-08-02)
  
  What does "backprop" mean? Is the "backprop" term basically the same as "backpropagation" or does it have a different meaning?
  
  **Tags:** neural-networks, backpropagation, terminology, definitions
  
  
  ---
  
  ## 5 answers
  
  ### Answer by username_7 on 2016-08-02. Score: 15 | Accepted answer
  
  "Backprop" is the same as "backpropagation": it's just a shorter way to say it. It is sometimes abbreviated as "BP".
  
  
  ---
  
  ### Answer by username_0 on 2016-08-03. Score: 10
  
  'Backprop' is short for 'backpropagation of error' in order to avoid confusion when using *backpropagation* term.
  
  Basically *backpropagation* refers to the method for computing the gradient of the case-wise error function with respect to the weights for a feedforward networkWerbos. And *backprop* refers to a training method that uses backpropagation to compute the gradient.
  
  So we can say that a *backprop* network is a feedforward network trained by *backpropagation*.
  
  The 'standard backprop' term is a euphemism for the *generalized delta rule* which is most widely used supervised training method.
  
  Source: [What is backprop?](ftp://ftp.sas.com/pub/neural/FAQ2.html#A_backprop) at FAQ of Usenet newsgroup comp.ai.neural-nets
  
  References:
  
  * Werbos, P. J. (1974). Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. PhD thesis, Harvard University.
  * Werbos, P. J. (1994). The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting,Wiley Interscience.
  * Bertsekas, D. P. (1995), Nonlinear Programming, Belmont, MA: Athena Scientific, ISBN 1-886529-14-0.
  * Bertsekas, D. P. and Tsitsiklis, J. N. (1996), Neuro-Dynamic Programming, Belmont, MA: Athena Scientific, ISBN 1-886529-10-8.
  * Polyak, B.T. (1964), "Some methods of speeding up the convergence of iteration methods," Z. Vycisl. Mat. i Mat. Fiz., 4, 1-17.
  * Polyak, B.T. (1987), Introduction to Optimization, NY: Optimization Software, Inc.
  * Reed, R.D., and Marks, R.J, II (1999), Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks, Cambridge, MA: The MIT Press, ISBN 0-262-18190-8.
  * Rumelhart, D.E., Hinton, G.E., and Williams, R.J. (1986), "Learning internal representations by error propagation", in Rumelhart, D.E. and McClelland, J. L., eds. (1986), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1, 318-362, Cambridge, MA: The MIT Press.
  * Werbos, P.J. (1974/1994), The Roots of Backpropagation, NY: John Wiley & Sons. Includes Werbos's 1974 Harvard Ph.D. thesis, Beyond Regression.
  
  > **Comments:**
  > **username_1**: maybe you need to accept your own answer.  
  
  ---
  
  ### Answer by username_2 on 2016-08-02. Score: 3
  
  Yes, as username_7 has rightly put, "backprop" means backpropogation, which is frequently used in the domain of neural networks for error optimization.
  
  For a detailed explanation, I would point out [this tutorial](http://neuralnetworksanddeeplearning.com/chap2.html) on the concept of backpropogation by a very good book of Michael Nielsen.
  
  
  ---
  
  ### Answer by username_3 on 2020-04-22. Score: 1
  
  It's a fancy name for the multivariable chain rule.
  
  > **Comments:**
  > **username_4**: Hello.  Could you support this answer with a little more detail?  (I realize it's a simple question, but I think your answer is worthy of greater explication.)  
  > **username_5**: @username_3 Please, follow the suggestion in the previous comment. At least, provide a link to an external resource, such as Wiki, that provides more details. Take a look at https://ai.stackexchange.com/help/how-to-answer to know more about what we expect from answers and how to write a good answer.  
  
  ---
  
  ### Answer by username_6 on 2021-07-08. Score: 0
  
  We need to compute the gradients in-order to train the deep neural networks. Deep neural network consists of many layers. Weight parameters are present between the layers. Since we need to compute the gradients of loss function for each weight, we use an algorithm called backprop. It is an abbreviation for **backprop**agation, which is also called as error backpropagation or reverse differentiation.
  
  It can be understood well from the following paragraph taken from [Neural Networks and Neural Language Models](https://web.stanford.edu/%7Ejurafsky/slp3/7.pdf)
  
  > For deep networks, computing the gradients for each weight is much
  > more complex,since we are computing the derivative with respect to
  > weight parameters that appear all the way back in the very early
  > layers of the network, even though the loss is computed only at the
  > very end of the network.**The solution to computing this gradient is an
  > algorithm called error backpropagation or backprop**. While backprop was
  > invented for neural networks, it turns out to be the same as a more
  > general procedure called backward differentiation, which depends on
  > the notion of computation graphs.
  
  
  ---
  
  