Answers.Body: <p>Sorry for the delay.
  The term "vector-valued feedback" is compared to scalar-valued feedback. The implication (which I should have made explicit) is that, because vector-valued feedback tells the network the correct answer, the changes in weights required to improve performance are reasonably easy to calculate (e.g. using backprop). </p>
  
  <p>In contrast, if a scalar-valued feedback is given (as in reinforcement learning) then the network knows only how bad its previous output was, but not how to change weights in order to improve the output.</p>
  
  <p>A rough analogy would be that vector-valued feedback tells you that you got the wrong answer to a question, and provides the correct answer. In contrast, scalar-valued feedback just tells you 'how wrong' your answer was, but does not tell you how to improve your answer.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-24T08:18:38.857Z
Answers.LastActivityDate: 2020-01-24T12:34:28.187Z
Answers.LastEditDate: 2020-01-24T12:34:28.187Z
Answers.UserDisplayName: James V Stone
Answers.Body: <p>By "immediate vector-valued feedback", they probably mean exactly the label in the "labeled examples" you mentioned.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-02T19:38:33.423Z
Answers.Text: Thanks. It does make sense (model dependency of the feedback sounded wrong to me too). Does this mean that, using reinforcement terms, in supervised learning we have the Agent, Action, and a static (single state) Environment, with the reward being immediate and explicit (vector-valued output is correct or not)? Thanks again.
Answers.UserDisplayName: ir7
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-02T20:33:08.257Z
Answers.Text: @ir7 This is a nice way to see it, but also I'd add that the environment is deterministic (whereas in general, in RL the environment is stochastic). Furthermore, using the other way around, you could see RL as a situation where you generate your labeled data dynamically.
Answers.UserDisplayName: olinarr
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-02T19:09:04.160Z
Answers.LastActivityDate: 2019-11-02T19:09:04.160Z
Answers.UserDisplayName: olinarr
Body: <p>In the book <a href="https://rads.stackoverflow.com/amzn/click/com/0956372813" rel="nofollow noreferrer" rel="nofollow noreferrer">Artificial Intelligence Engines: A Tutorial Introduction to the Mathematics of Deep Learning</a>, James Stone says</p>
  
  <blockquote>
    <p>With supervised learning, the response to each input vector is an output vector that <strong>receives immediate vector-valued feedback specifying the correct output</strong>, and this feedback refers uniquely to the input vector just received; in contrast, each reinforcement learning output vector (action) receives scalar-valued feedback often sometime after the action, and this feedback signal depends on actions taken before and after the current action.</p>
  </blockquote>
  
  <p>I fail to understand the part formatted in bold. Once we have a set of labeled examples (feature vector and label pairs), where is the "feedback" coming from? Testing and validation results of our calibrated model (say a neural network based one)? </p>
  
Tags: <machine-learning><terminology><supervised-learning>
Title: What does "immediate vector-valued feedback" mean?
UserDisplayName: ir7
ThreadText: # What does "immediate vector-valued feedback" mean?
  
  (Asked by: username_0 on 2019-11-02)
  
  In the book [Artificial Intelligence Engines: A Tutorial Introduction to the Mathematics of Deep Learning](https://rads.stackoverflow.com/amzn/click/com/0956372813), James Stone says
  
  > With supervised learning, the response to each input vector is an output vector that **receives immediate vector-valued feedback specifying the correct output**, and this feedback refers uniquely to the input vector just received; in contrast, each reinforcement learning output vector (action) receives scalar-valued feedback often sometime after the action, and this feedback signal depends on actions taken before and after the current action.
  
  I fail to understand the part formatted in bold. Once we have a set of labeled examples (feature vector and label pairs), where is the "feedback" coming from? Testing and validation results of our calibrated model (say a neural network based one)?
  
  **Tags:** machine-learning, terminology, supervised-learning
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2019-11-02. Score: 2 | Accepted answer
  
  By "immediate vector-valued feedback", they probably mean exactly the label in the "labeled examples" you mentioned.
  
  > **Comments:**
  > **username_0**: Thanks. It does make sense (model dependency of the feedback sounded wrong to me too). Does this mean that, using reinforcement terms, in supervised learning we have the Agent, Action, and a static (single state) Environment, with the reward being immediate and explicit (vector-valued output is correct or not)? Thanks again.  
  > **username_2**: @ir7 This is a nice way to see it, but also I'd add that the environment is deterministic (whereas in general, in RL the environment is stochastic). Furthermore, using the other way around, you could see RL as a situation where you generate your labeled data dynamically.  
  
  ---
  
  ### Answer by username_1 on 2020-01-24. Score: 1
  
  Sorry for the delay.
  The term "vector-valued feedback" is compared to scalar-valued feedback. The implication (which I should have made explicit) is that, because vector-valued feedback tells the network the correct answer, the changes in weights required to improve performance are reasonably easy to calculate (e.g. using backprop).
  
  In contrast, if a scalar-valued feedback is given (as in reinforcement learning) then the network knows only how bad its previous output was, but not how to change weights in order to improve the output.
  
  A rough analogy would be that vector-valued feedback tells you that you got the wrong answer to a question, and provides the correct answer. In contrast, scalar-valued feedback just tells you 'how wrong' your answer was, but does not tell you how to improve your answer.
  
  
  ---
  
  