Answers.Body: <p>I would say that these concepts are quite different, even though they might have a few things in common (or might be vaguely related). </p>
  
  <p><a href="https://ai.stackexchange.com/a/8985/2444">Back-propagation</a> is an <em>algorithm</em> used (in machine learning) to compute the gradient of a function with respect to its parameters. This gradient is then used by algorithms, like gradient descent, to update the parameters of the model (e.g. a neural network).</p>
  
  <p>Roughly, <a href="https://theness.com/neurologicablog/index.php/the-brain-and-predictive-coding/" rel="nofollow noreferrer">predictive coding</a> is a general (neuroscience) <em>theory</em> of how the brain builds an internal model of the external world, of how it continuously predicts the sensory inputs (from the world) using its current model of the external world, and of how it updates this internal model once the sensory inputs are actually received.</p>
  
  <p>You could think of the output of an ML model, while it is being trained (using gradient descent with back-propagation), as a prediction associated with the inputs. However, note that the output of the model is often not a prediction of the actual input, but e.g. a label (which is, nonetheless, associated with the input). Furthermore, it is the model that produces an output and not the back-propagation algorithm (even though the back-propagation algorithm is often used to train such models, e.g. neural networks). We could think of back-propagation as the way of updating these predictions associated with the inputs, but, anyway, it is well known (and accepted) that our brain does not perform back-propagation, but we learn in an <a href="https://en.wikipedia.org/wiki/Hebbian_theory" rel="nofollow noreferrer">associative fashion</a> (Hebbian learning).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-11T16:35:30.547Z
Answers.LastActivityDate: 2019-03-11T16:52:11.107Z
Answers.LastEditDate: 2019-03-11T16:52:11.107Z
Answers.UserDisplayName: nbro
Body: <p>Reading the high-level descriptions of <a href="https://en.wikipedia.org/wiki/Backpropagation" rel="nofollow noreferrer">backpropagation</a> and <a href="https://en.wikipedia.org/wiki/Predictive_coding" rel="nofollow noreferrer">predictive coding</a>, they don't sound so drastically different. What is the key difference between these techniques?</p>
  
  <p>I am currently reading the following paper if that helps ground the explanation:</p>
  
  <p><a href="https://arxiv.org/pdf/1706.02444.pdf" rel="nofollow noreferrer">Predictive Coding-based Deep Dynamic Neural Network for Visuomotor
  Learning</a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-04T03:24:53.480Z
Comments.Text: In this paper, the author proved that [backpropagation](https://www.mitpressjournals.org/doi/full/10.1162/NECO_a_00949) is a special case of predictive coding.
Comments.UserDisplayName: Lerner Zhang
Tags: <neural-networks><machine-learning><backpropagation><comparison>
Title: What is the difference between backpropagation and predictive coding?
UserDisplayName: adamconkey
ThreadText: # What is the difference between backpropagation and predictive coding?
  
  (Asked by: username_0 on 2019-03-11)
  
  Reading the high-level descriptions of [backpropagation](https://en.wikipedia.org/wiki/Backpropagation) and [predictive coding](https://en.wikipedia.org/wiki/Predictive_coding), they don't sound so drastically different. What is the key difference between these techniques?
  
  I am currently reading the following paper if that helps ground the explanation:
  
  [Predictive Coding-based Deep Dynamic Neural Network for Visuomotor
  Learning](https://arxiv.org/pdf/1706.02444.pdf)
  
  **Tags:** neural-networks, machine-learning, backpropagation, comparison
  > **Comments:**
  > **username_1**: In this paper, the author proved that [backpropagation](https://www.mitpressjournals.org/doi/full/10.1162/NECO_a_00949) is a special case of predictive coding.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-03-11. Score: 1
  
  I would say that these concepts are quite different, even though they might have a few things in common (or might be vaguely related).
  
  [Back-propagation](https://ai.stackexchange.com/a/8985/2444) is an *algorithm* used (in machine learning) to compute the gradient of a function with respect to its parameters. This gradient is then used by algorithms, like gradient descent, to update the parameters of the model (e.g. a neural network).
  
  Roughly, [predictive coding](https://theness.com/neurologicablog/index.php/the-brain-and-predictive-coding/) is a general (neuroscience) *theory* of how the brain builds an internal model of the external world, of how it continuously predicts the sensory inputs (from the world) using its current model of the external world, and of how it updates this internal model once the sensory inputs are actually received.
  
  You could think of the output of an ML model, while it is being trained (using gradient descent with back-propagation), as a prediction associated with the inputs. However, note that the output of the model is often not a prediction of the actual input, but e.g. a label (which is, nonetheless, associated with the input). Furthermore, it is the model that produces an output and not the back-propagation algorithm (even though the back-propagation algorithm is often used to train such models, e.g. neural networks). We could think of back-propagation as the way of updating these predictions associated with the inputs, but, anyway, it is well known (and accepted) that our brain does not perform back-propagation, but we learn in an [associative fashion](https://en.wikipedia.org/wiki/Hebbian_theory) (Hebbian learning).
  
  
  ---
  
  