Answers.Body: <p>Any supervised learning (SL) problem can be cast as an equivalent reinforcement learning (RL) one.</p>
  <p>Suppose you have the training dataset <span class="math-container">$\mathcal{D} = \{ (x_i, y_i \}_{i=1}^N$</span>, where <span class="math-container">$x_i$</span> is an observation and <span class="math-container">$y_i$</span> the corresponding label. Then let <span class="math-container">$x_i$</span> be a state and let <span class="math-container">$f(x_i) = \hat{y}_i$</span>, where <span class="math-container">$f$</span> is your (current) model, be an action. So, the predicted label of observation <span class="math-container">$x_i$</span> corresponds to the action taken in state <span class="math-container">$x_i$</span>. The reward received after having taken action <span class="math-container">$f(x_i)$</span> in state <span class="math-container">$x_i$</span> can then be defined as the loss <span class="math-container">$|f(x_i) - y_i|$</span> (or any other suitable loss).</p>
  <p>The minimization of this loss is then equivalent to the maximization of the (expected) reward. Therefore, in theory, you could use trajectories of the form <span class="math-container">$$T=\{(x_1, f(x_1), |f(x_1) - y_1|),  \dots, (x_N, f(x_N), |f(x_N) - y_N|)\}$$</span> to learn a value function <span class="math-container">$q$</span> (for example, with Q-learning) or a policy <span class="math-container">$\pi$</span>, which then, given a new state <span class="math-container">$x_{\text{new}}$</span> (an observation) produces an action <span class="math-container">$f(x_{\text{new}})$</span> (the predicted label).</p>
  <p>However, note that the learned policy might not be able to generalise to observations not present in the training dataset. Moreover, although it is possible to solve an SL problem as an RL problem, in practice, this may not be the most appropriate approach (i.e. it may be inefficient).</p>
  <p>For more details, read the paper <a href="https://pdfs.semanticscholar.org/ccec/df227475db9be6f0fa8f431c1db51e9dcd83.pdf" rel="noreferrer">Reinforcement Learning and its Relationship to Supervised Learning</a> (2004) by Barto and Dietterich, who give a good overview of supervised and reinforcement learning and their relationship. The paper <a href="https://link.springer.com/article/10.1007/BF00115009" rel="noreferrer">Learning to predict by the methods of temporal differences</a> (1988), by Richard Sutton, should also give you an overview of reinforcement learning from a supervised learning perspective. However, note that this does not mean that a reinforcement learning problem can be cast as an equivalent supervised learning one. See section <em>1.3.3 Converting Reinforcement Learning to Supervised Learning</em> of the mentioned paper <a href="https://pdfs.semanticscholar.org/ccec/df227475db9be6f0fa8f431c1db51e9dcd83.pdf" rel="noreferrer">Reinforcement Learning and its Relationship to Supervised Learning</a> for more details.</p>
  <p>Reinforcement learning can thus be used for classification and regression tasks. See, for example, <a href="http://www.maths.lth.se/sminchisescu/media/papers/mps-rlcvpr16.pdf" rel="noreferrer">Reinforcement Learning for Visual Object Detection</a> (2016) by Mathe et al.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-27T23:14:02.027Z
Answers.LastActivityDate: 2020-07-07T15:27:16.643Z
Answers.LastEditDate: 2020-07-07T15:27:16.643Z
Answers.UserDisplayName: nbro
Body: <p>Let's assume that there is a sequence of pairs <span class="math-container">$(x_i, y_i), (x_{i+1}, y_{i+1}), \dots$</span> of observations and corresponding labels. Let's also assume that the <span class="math-container">$x$</span> is considered as independent variable and <span class="math-container">$y$</span> is considered as the variable that depends on <span class="math-container">$x$</span>. So, in supervised learning, one wants to learn the function <span class="math-container">$y=f(x)$</span>. </p>
  
  <p>Can reinforcement learning be used to learn <span class="math-container">$f$</span> (possibly, even learning the symbolic form of <span class="math-container">$f(x)$</span>)? </p>
  
  <p>Just some sketches how can it be done: <span class="math-container">$x_i$</span> can be considered as the environment and each <span class="math-container">$x_i$</span> defines some set of possible "actions" - possible symbolic form of <span class="math-container">$f(x)$</span> or possible numerical values of parameters for <span class="math-container">$f(x)$</span> (if the symbolic form is fized). And concrete selected action/functional form <span class="math-container">$f(x, a)$</span> (a - set of parameters) can be assigned reward from the loss function: how close the observation <span class="math-container">$(x_i, y_i)$</span> is to the value that can be inferred from <span class="math-container">$f(x)$</span>.</p>
  
  <p>Are there ideas or works of RL along the framework that I provided in the previous passage?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-12T00:19:12.503Z
Comments.Text: Interestingly, the opposite is also true: reinforcement learning can be cast as supervised learning! See for example https://arxiv.org/abs/1912.02875
Comments.UserDisplayName: mdc
Tags: <reinforcement-learning><comparison><supervised-learning><function-approximation><regression>
Title: Can supervised learning be recast as reinforcement learning problem?
UserDisplayName: TomR
ThreadText: # Can supervised learning be recast as reinforcement learning problem?
  
  (Asked by: username_0 on 2019-08-27)
  
  Let's assume that there is a sequence of pairs $(x\_i, y\_i), (x\_{i+1}, y\_{i+1}), \dots$ of observations and corresponding labels. Let's also assume that the $x$ is considered as independent variable and $y$ is considered as the variable that depends on $x$. So, in supervised learning, one wants to learn the function $y=f(x)$.
  
  Can reinforcement learning be used to learn $f$ (possibly, even learning the symbolic form of $f(x)$)?
  
  Just some sketches how can it be done: $x\_i$ can be considered as the environment and each $x\_i$ defines some set of possible "actions" - possible symbolic form of $f(x)$ or possible numerical values of parameters for $f(x)$ (if the symbolic form is fized). And concrete selected action/functional form $f(x, a)$ (a - set of parameters) can be assigned reward from the loss function: how close the observation $(x\_i, y\_i)$ is to the value that can be inferred from $f(x)$.
  
  Are there ideas or works of RL along the framework that I provided in the previous passage?
  
  **Tags:** reinforcement-learning, comparison, supervised-learning, function-approximation, regression
  
  > **Comments:**
  > **username_1**: Interestingly, the opposite is also true: reinforcement learning can be cast as supervised learning! See for example https://arxiv.org/abs/1912.02875  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-08-27. Score: 16 | Accepted answer
  
  Any supervised learning (SL) problem can be cast as an equivalent reinforcement learning (RL) one.
  
  Suppose you have the training dataset $\mathcal{D} = \{ (x\_i, y\_i \}\_{i=1}^N$, where $x\_i$ is an observation and $y\_i$ the corresponding label. Then let $x\_i$ be a state and let $f(x\_i) = \hat{y}\_i$, where $f$ is your (current) model, be an action. So, the predicted label of observation $x\_i$ corresponds to the action taken in state $x\_i$. The reward received after having taken action $f(x\_i)$ in state $x\_i$ can then be defined as the loss $|f(x\_i) - y\_i|$ (or any other suitable loss).
  
  The minimization of this loss is then equivalent to the maximization of the (expected) reward. Therefore, in theory, you could use trajectories of the form $$T=\{(x\_1, f(x\_1), |f(x\_1) - y\_1|), \dots, (x\_N, f(x\_N), |f(x\_N) - y\_N|)\}$$ to learn a value function $q$ (for example, with Q-learning) or a policy $\pi$, which then, given a new state $x\_{\text{new}}$ (an observation) produces an action $f(x\_{\text{new}})$ (the predicted label).
  
  However, note that the learned policy might not be able to generalise to observations not present in the training dataset. Moreover, although it is possible to solve an SL problem as an RL problem, in practice, this may not be the most appropriate approach (i.e. it may be inefficient).
  
  For more details, read the paper [Reinforcement Learning and its Relationship to Supervised Learning](https://pdfs.semanticscholar.org/ccec/df227475db9be6f0fa8f431c1db51e9dcd83.pdf) (2004) by Barto and Dietterich, who give a good overview of supervised and reinforcement learning and their relationship. The paper [Learning to predict by the methods of temporal differences](https://link.springer.com/article/10.1007/BF00115009) (1988), by Richard Sutton, should also give you an overview of reinforcement learning from a supervised learning perspective. However, note that this does not mean that a reinforcement learning problem can be cast as an equivalent supervised learning one. See section *1.3.3 Converting Reinforcement Learning to Supervised Learning* of the mentioned paper [Reinforcement Learning and its Relationship to Supervised Learning](https://pdfs.semanticscholar.org/ccec/df227475db9be6f0fa8f431c1db51e9dcd83.pdf) for more details.
  
  Reinforcement learning can thus be used for classification and regression tasks. See, for example, [Reinforcement Learning for Visual Object Detection](http://www.maths.lth.se/sminchisescu/media/papers/mps-rlcvpr16.pdf) (2016) by Mathe et al.
  
  
  ---
  
  