Answers.Body: <h1>Reinforcement Learning Workflow</h1>
  
  <p>The general workflow for using and applying reinforcement learning to solve a task is the following.</p>
  
  <p><a href="https://i.sstatic.net/RV6cf.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/RV6cf.jpg" alt="enter image description here"></a></p>
  
  <ol>
  <li><strong>Create the Environment</strong></li>
  <li><strong>Define the Reward</strong></li>
  <li><strong>Create the Agent</strong></li>
  <li><strong>Train and Validate the Agent</strong></li>
  <li><strong>Deploy the Policy</strong></li>
  </ol>
  
  <hr>
  
  <h1>Training</h1>
  
  <ul>
  <li><p>Training in Reinforcement learning employs a <strong>system of rewards</strong> and <strong>penalties</strong> to compel the computer to solve a problem by itself.</p></li>
  <li><p>Human involvement is <strong>limited</strong> to <strong>changing</strong> the environment and <strong>tweaking</strong> the system of rewards and penalties.</p></li>
  <li><p>As the computer <strong>maximizes the reward</strong>, it is prone to seeking unexpected ways of doing it. </p></li>
  <li><p>Human involvement is focused on <strong>preventing</strong> it from <strong>exploiting the system</strong> and <strong>motivating</strong> the machine to perform the task in the way expected.</p></li>
  <li><p>Reinforcement learning is useful when there is no <strong>“proper way”</strong> to perform a task, yet there are rules the model has to follow to perform its duties correctly.</p></li>
  <li><p>Example: By tweaking and seeking the optimal policy for deep reinforcement learning, we built an agent that in just <strong>20 minutes</strong> reached a superhuman level in playing <strong>Atari games</strong>.</p></li>
  <li><p>Similar algorithms, in principle, can be used to build AI for an <strong>autonomous car.</strong></p></li>
  </ul>
  
  <p><a href="https://i.sstatic.net/lplhT.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/lplhT.png" alt="Car"></a></p>
  
  <h1>Testing</h1>
  
  <ul>
  <li><p>Debugging RL algorithms is very hard. Everything runs and you are not sure where the problem is.</p></li>
  <li><p>To test if it worked well, if the trained agent is good at what it was trained for, <strong>you take your trained model and apply it to the situation it is trained for</strong>.</p></li>
  <li><p>If it’s something like <strong>chess</strong> or <strong>Go</strong>, you could <strong>benchmark</strong> it against other engines (say stockfish for chess) or human players.</p></li>
  <li><p>You can also define metrics for performance, ways of measuring the quality of the agent’s decisions. </p></li>
  <li><p>In some settings (e.g a <strong>Reinforcement Learning Pacman player</strong>), the game score literally defines the target outcome, so you can just evaluate your model’s performance based on that metric.</p></li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-05T16:23:06.487Z
Answers.LastActivityDate: 2020-05-05T22:08:17.043Z
Answers.LastEditDate: 2020-05-05T22:08:17.043Z
Answers.UserDisplayName: Faizy
Answers.Body: <p>The goal of the reinforcement learning (RL) is to use data obtained via interaction with the environment to <strong>solve the underlying Markov Decision Process</strong> (MDP). "Solving the MDP" is tantamount to finding the optimal policy (with respect to the MDP's underlying dynamics which are usually assumed to be stationary). </p>
  
  <p><strong>Training</strong> is the process of using data in order to find the optimal policy. 
  <strong>Testing</strong> is the process of evaluating the (final) policy obtained by training.</p>
  
  <p>Note that, since we're generally testing the policy on the same MDP we used for training, the <strong>distinction between the training dataset and the testing set</strong> is no longer as important as it is the case with say supervised learning. Consequently, classical notions of <strong>overfitting and generalization</strong> should be approached from a different angle as well.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-05T16:47:43.480Z
Answers.LastActivityDate: 2020-05-05T16:47:43.480Z
Answers.UserDisplayName: BGasperov
Answers.Body: <p>If you want, you can do training and testing in RL.
  Exactly the same usage, training for building up a policy, and testing for evaluation.</p>
  <p>In supervised learning, if you use test data in training, it is like cheating. You cannot trust the evaluation. That's why we separate train and test data.</p>
  <p>The Objective of RL is a little different.
  RL trying to find the optimal policy. Since RL collects the information by doing, while the agent explores the environment (for more information), there might be a loss in the objective function. But, it might be inevitable for a better future gain.</p>
  <p>Multi-arm bandit example,  If there are 10 slot machines. They will return random amounts of money. They have different expected returns. I want to find the best way to maximize my gain. easy, I have to find the machine with the greatest expected return and use only the machine. How to find the best machine?</p>
  <p>If we have a training and testing (periods),
  For example, I will give you an hour of the training period, so it doesn't matter if you lose or how much you earn. And in the testing period, I will evaluate your performance.</p>
  <p>What would you do? In the training period, you will try as much as possible, without considering the performance/gain. And in the testing period, you will use only the best machine you found.</p>
  <p>This is not a typical RL situation. RL is trying to find the best way, Learning by doing. All the results while doing are considered.</p>
  <p>Suppose that I tried all 10 machines once each. And, the No.3 machine gave me the most money. But I am not sure that it is the best machine, because all the machines provide a RANDOM amount. If I keep using the No.3 machine, it might be a good idea, because according to the information so far, it is the best machine. However, You might miss the better machine if you don't try other machines due to randomness. But if you try other machines, you might lose an opportunity to earn more money.
  What should I do?  This is a well-known Exploration and Exploitation trade-off in RL.</p>
  <p>RL trying to maximize the gain including the gains right now and the gains in the future.
  In other words, the performance during training is also considered as its performance.
  That's why RL is not unsupervised nor supervised learning.</p>
  <p>However, in some situations, you might want to separate training and testing. RL is designed for an agent who interacts with the environment. However,  in some cases, (for example), rather than having an interactive playground, you have data of interactions. The formulation would be a little different in this case.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-05T15:07:04.587Z
Answers.LastActivityDate: 2021-09-29T07:09:17.950Z
Answers.LastEditDate: 2021-09-29T07:09:17.950Z
Answers.UserDisplayName: S. Phil Kim
Answers.Body: <h1>What is reinforcement learning?</h1>
  <p>In reinforcement learning (RL), you typically imagine that there's an agent that interacts, in time steps, with an environment by taking actions. On each time step <span class="math-container">$t$</span>, the agent takes the action <span class="math-container">$a_t \in \mathcal{A}$</span> in the state <span class="math-container">$s_t \in \mathcal{S}$</span>, receives a reward (or reinforcement) signal <span class="math-container">$r_t \in \mathbb{R}$</span> from the environment and the agent and the environment move to another state <span class="math-container">$s_{t+1} \in \mathcal{S}$</span>, where <span class="math-container">$\mathcal{A}$</span> is the action space and <span class="math-container">$\mathcal{S}$</span> is the state space of the environment, which is typically assumed to be a Markov decision process (MDP).</p>
  <h1>What is the goal in RL?</h1>
  <p>The goal is to find a policy that maximizes the <strong>expected return</strong>
  (i.e. a sum of rewards starting from the current time step). The policy that maximizes the expected return is called the <strong>optimal policy</strong>.</p>
  <h2>Policies</h2>
  <p>A policy is a function that maps states to actions. Intuitively, the policy is the strategy that implements the behavior of the RL agent while interacting with the environment.</p>
  <p>A policy can be deterministic or stochastic. A deterministic policy can be denoted as <span class="math-container">$\pi : \mathcal{S} \rightarrow \mathcal{A}$</span>. So, a deterministic policy maps a state <span class="math-container">$s$</span> to an action <span class="math-container">$a$</span> with probability <span class="math-container">$1$</span>. A stochastic policy maps states to a probability distribution over actions. A stochastic policy can thus be denoted as <span class="math-container">$\pi(a \mid s)$</span> to indicate that it is a conditional probability distribution of an action <span class="math-container">$a$</span> given that the agent is in the state <span class="math-container">$s$</span>.</p>
  <h2>Expected return</h2>
  <p>The expected return can be formally written as</p>
  <p><span class="math-container">$$\mathbb{E}\left[ G_t \right] = \mathbb{E}\left[ \sum_{i=t+1}^\infty R_i \right]$$</span></p>
  <p>where <span class="math-container">$t$</span> is the current time step (so we don't care about the past), <span class="math-container">$R_i$</span> is a random variable that represents the probable reward at time step <span class="math-container">$i$</span>, and <span class="math-container">$G_t = \sum_{i=t+1}^\infty R_i $</span> is the so-called <em>return</em> (i.e. a sum of future rewards, in this case, starting from time step <span class="math-container">$t$</span>), which is also a random variable.</p>
  <h2>Reward function</h2>
  <p>In this context, the most important job of the human programmer is to define a function <span class="math-container">$\mathcal{R}: \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$</span>, the reward function, which provides the <em>reinforcement</em> (or reward) signal to the RL agent while interacting with the environment. <span class="math-container">$\mathcal{R}$</span> will deterministically or stochastically determine the reward that the agent receives every time it takes action <span class="math-container">$a$</span> in the state <span class="math-container">$s$</span>. The reward function <span class="math-container">$R$</span> is also part of the environment (i.e. the MDP).</p>
  <p>Note that <span class="math-container">$\mathcal{R}$</span>, the reward function, is different from <span class="math-container">$R_i$</span>, which is a random variable that represents the reward at time step <span class="math-container">$i$</span>. However, clearly, the two are very related. In fact, the reward function will determine the actual <em>realizations</em> of the random variables <span class="math-container">$R_i$</span> and thus of the return <span class="math-container">$G_i$</span>.</p>
  <h2>How to estimate the optimal policy?</h2>
  <p>To estimate the optimal policy, you typically design optimization algorithms.</p>
  <h3>Q-learning</h3>
  <p>The most famous RL algorithm is probably Q-learning, which is also a numerical and iterative algorithm. Q-learning implements the interaction between an RL agent and the environment (described above). More concretely, it attempts to estimate a function that is closely related to the policy and from which the policy can be derived. This function is called the <strong>value function</strong>, and, in the case of Q-learning, it's a function of the form <span class="math-container">$Q : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$</span>. The name <span class="math-container">$Q$</span>-learning derives from this function, which is often denoted as <span class="math-container">$Q$</span>.</p>
  <p>Q-learning doesn't necessarily find the optimal policy, but there are cases where it is guaranteed to find the optimal policy (but I won't dive into the details).</p>
  <p>Of course, I cannot describe all the details of Q-learning in this answer. Just keep in mind that, to estimate a policy, in RL, you will typically use a numerical and iterative optimization algorithm (e.g. Q-learning).</p>
  <h2>What is training in RL?</h2>
  <p>In RL, training (also known as <em>learning</em>) generally refers to the use of RL algorithms, such as Q-learning, to estimate the optimal policy (or a value function)</p>
  <p>Of course, as in any other machine learning problem (such as supervised learning), there are many practical considerations related to the implementation of these RL algorithms, such as</p>
  <ul>
  <li>Which RL algorithm to use?</li>
  <li>Which programming language, library, or framework to use?</li>
  </ul>
  <p>These and other details (which, of course, I cannot list exhaustively) can actually affect the policy that you obtain. However, the basic goal during the learning or training phase in RL is to find a policy (possibly, optimal, but this is almost never the case).</p>
  <h2>What is evaluation (or testing) in RL?</h2>
  <p>During learning (or training), you may not be able to find the optimal policy, so how can you be sure that the learned policy to solve the actual real-world problem is <em>good enough</em>? This question needs to be answered, ideally before deploying your RL algorithm.</p>
  <p>The evaluation phase of an RL algorithm is the <strong>assessment of the quality of the learned policy</strong> and <strong>how much reward the agent obtains if it follows that policy</strong>. So, a typical metric that can be used to assess the quality of the policy is to plot <strong>the sum of all rewards received so far</strong> (i.e. cumulative reward or return) <strong>as a function of the number of steps</strong>. One RL algorithm dominates another if its plot is consistently above the other. You should note that the evaluation phase can actually occur during the training phase too. Moreover, you could also assess the <strong>generalization</strong> of your learned policy by evaluating it (as just described) in different (but similar) environments to the training environment [<a href="http://proceedings.mlr.press/v97/cobbe19a/cobbe19a.pdf" rel="nofollow noreferrer">1</a>].</p>
  <p>The section <a href="https://artint.info/2e/html/ArtInt2e.Ch12.S6.html" rel="nofollow noreferrer">12.6 Evaluating Reinforcement Learning Algorithms</a> of the book <a href="https://artint.info/2e/html/ArtInt2e.html" rel="nofollow noreferrer">Artificial Intelligence: Foundations of Computational Agents</a> (2017) by Poole and Mackworth provides more details about the evaluation phase in reinforcement learning, so you should probably read it.</p>
  <p>Apart from evaluating the learned policy, you can also evaluate your RL algorithm, in terms of</p>
  <ul>
  <li><strong>resources used</strong> (such as CPU and memory), and/or</li>
  <li>experience/data/samples needed to converge to a certain level of performance (i.e. you can evaluate the <strong>data/sample efficiency</strong> of your RL algorithm)</li>
  <li><strong>robustness/sensitivity</strong> (i.e., how the RL algorithm behaves if you change certain hyper-parameters); this is also important because RL algorithms can be very sensitive (from my experience)</li>
  </ul>
  <h2>What is the difference between training and evaluation?</h2>
  <p>During training, you want to find the policy. During the evaluation, you want to assess the quality of the learned policy (or RL algorithm). You can perform the evaluation even during training.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-27T17:19:41.210Z
Answers.Text: [Here](https://groups.google.com/g/rl-list/c/7EHj4rtjY9I/m/ezLss7vV7wEJ) is another answer that is worth reading too.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-05T23:40:33.497Z
Answers.LastActivityDate: 2020-11-23T15:42:19.127Z
Answers.LastEditDate: 2020-11-23T15:42:19.127Z
Answers.UserDisplayName: nbro
Body: <p>In reinforcement learning (RL), what is the difference between training and testing an algorithm/agent? If I understood correctly, testing is also referred to as evaluation.</p>
  
  <p>As I see it, both imply the same procedure: select an action, apply to the environment, get a reward, and next state, and so on. But I've seen that, e.g., the <a href="https://github.com/tensorforce/tensorforce" rel="nofollow noreferrer">Tensorforce RL framework</a> allows running with or without evaluation.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-04T15:54:26.073Z
Comments.Text: As RL‌ is not a supervised algorithm (it's a third type of ML algorithms), you can't the same expectation from testing and training algorithm here, as well as, supervised algorithms.
Comments.UserDisplayName: OmG
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-04T16:03:31.657Z
Comments.Text: @OmG OK. So, as I understand from you, this concept does not apply to RL?
Comments.UserDisplayName: Cristian M
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-04T17:31:11.457Z
Comments.Text: Not the same as the supervised learning.
Comments.UserDisplayName: OmG
Tags: <reinforcement-learning><training><comparison><testing>
Title: What is the difference between training and testing in reinforcement learning?
UserDisplayName: Cristian M
ThreadText: # What is the difference between training and testing in reinforcement learning?
  
  (Asked by: username_0 on 2020-05-04)
  
  In reinforcement learning (RL), what is the difference between training and testing an algorithm/agent? If I understood correctly, testing is also referred to as evaluation.
  
  As I see it, both imply the same procedure: select an action, apply to the environment, get a reward, and next state, and so on. But I've seen that, e.g., the [Tensorforce RL framework](https://github.com/tensorforce/tensorforce) allows running with or without evaluation.
  
  **Tags:** reinforcement-learning, training, comparison, testing
  > **Comments:**
  > **username_1**: As RL‌ is not a supervised algorithm (it's a third type of ML algorithms), you can't the same expectation from testing and training algorithm here, as well as, supervised algorithms.  
  > **username_0**: @OmG OK. So, as I understand from you, this concept does not apply to RL?  
  > **username_1**: Not the same as the supervised learning.  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_5 on 2020-05-05. Score: 8 | Accepted answer
  
  # What is reinforcement learning?
  
  In reinforcement learning (RL), you typically imagine that there's an agent that interacts, in time steps, with an environment by taking actions. On each time step $t$, the agent takes the action $a\_t \in \mathcal{A}$ in the state $s\_t \in \mathcal{S}$, receives a reward (or reinforcement) signal $r\_t \in \mathbb{R}$ from the environment and the agent and the environment move to another state $s\_{t+1} \in \mathcal{S}$, where $\mathcal{A}$ is the action space and $\mathcal{S}$ is the state space of the environment, which is typically assumed to be a Markov decision process (MDP).
  
  # What is the goal in RL?
  
  The goal is to find a policy that maximizes the **expected return**
  (i.e. a sum of rewards starting from the current time step). The policy that maximizes the expected return is called the **optimal policy**.
  
  ## Policies
  
  A policy is a function that maps states to actions. Intuitively, the policy is the strategy that implements the behavior of the RL agent while interacting with the environment.
  
  A policy can be deterministic or stochastic. A deterministic policy can be denoted as $\pi : \mathcal{S} \rightarrow \mathcal{A}$. So, a deterministic policy maps a state $s$ to an action $a$ with probability $1$. A stochastic policy maps states to a probability distribution over actions. A stochastic policy can thus be denoted as $\pi(a \mid s)$ to indicate that it is a conditional probability distribution of an action $a$ given that the agent is in the state $s$.
  
  ## Expected return
  
  The expected return can be formally written as
  
  $$\mathbb{E}\left[ G\_t \right] = \mathbb{E}\left[ \sum\_{i=t+1}^\infty R\_i \right]$$
  
  where $t$ is the current time step (so we don't care about the past), $R\_i$ is a random variable that represents the probable reward at time step $i$, and $G\_t = \sum\_{i=t+1}^\infty R\_i $ is the so-called *return* (i.e. a sum of future rewards, in this case, starting from time step $t$), which is also a random variable.
  
  ## Reward function
  
  In this context, the most important job of the human programmer is to define a function $\mathcal{R}: \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$, the reward function, which provides the *reinforcement* (or reward) signal to the RL agent while interacting with the environment. $\mathcal{R}$ will deterministically or stochastically determine the reward that the agent receives every time it takes action $a$ in the state $s$. The reward function $R$ is also part of the environment (i.e. the MDP).
  
  Note that $\mathcal{R}$, the reward function, is different from $R\_i$, which is a random variable that represents the reward at time step $i$. However, clearly, the two are very related. In fact, the reward function will determine the actual *realizations* of the random variables $R\_i$ and thus of the return $G\_i$.
  
  ## How to estimate the optimal policy?
  
  To estimate the optimal policy, you typically design optimization algorithms.
  
  ### Q-learning
  
  The most famous RL algorithm is probably Q-learning, which is also a numerical and iterative algorithm. Q-learning implements the interaction between an RL agent and the environment (described above). More concretely, it attempts to estimate a function that is closely related to the policy and from which the policy can be derived. This function is called the **value function**, and, in the case of Q-learning, it's a function of the form $Q : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$. The name $Q$-learning derives from this function, which is often denoted as $Q$.
  
  Q-learning doesn't necessarily find the optimal policy, but there are cases where it is guaranteed to find the optimal policy (but I won't dive into the details).
  
  Of course, I cannot describe all the details of Q-learning in this answer. Just keep in mind that, to estimate a policy, in RL, you will typically use a numerical and iterative optimization algorithm (e.g. Q-learning).
  
  ## What is training in RL?
  
  In RL, training (also known as *learning*) generally refers to the use of RL algorithms, such as Q-learning, to estimate the optimal policy (or a value function)
  
  Of course, as in any other machine learning problem (such as supervised learning), there are many practical considerations related to the implementation of these RL algorithms, such as
  
  * Which RL algorithm to use?
  * Which programming language, library, or framework to use?
  
  These and other details (which, of course, I cannot list exhaustively) can actually affect the policy that you obtain. However, the basic goal during the learning or training phase in RL is to find a policy (possibly, optimal, but this is almost never the case).
  
  ## What is evaluation (or testing) in RL?
  
  During learning (or training), you may not be able to find the optimal policy, so how can you be sure that the learned policy to solve the actual real-world problem is *good enough*? This question needs to be answered, ideally before deploying your RL algorithm.
  
  The evaluation phase of an RL algorithm is the **assessment of the quality of the learned policy** and **how much reward the agent obtains if it follows that policy**. So, a typical metric that can be used to assess the quality of the policy is to plot **the sum of all rewards received so far** (i.e. cumulative reward or return) **as a function of the number of steps**. One RL algorithm dominates another if its plot is consistently above the other. You should note that the evaluation phase can actually occur during the training phase too. Moreover, you could also assess the **generalization** of your learned policy by evaluating it (as just described) in different (but similar) environments to the training environment [[1](http://proceedings.mlr.press/v97/cobbe19a/cobbe19a.pdf)].
  
  The section [12.6 Evaluating Reinforcement Learning Algorithms](https://artint.info/2e/html/ArtInt2e.Ch12.S6.html) of the book [Artificial Intelligence: Foundations of Computational Agents](https://artint.info/2e/html/ArtInt2e.html) (2017) by Poole and Mackworth provides more details about the evaluation phase in reinforcement learning, so you should probably read it.
  
  Apart from evaluating the learned policy, you can also evaluate your RL algorithm, in terms of
  
  * **resources used** (such as CPU and memory), and/or
  * experience/data/samples needed to converge to a certain level of performance (i.e. you can evaluate the **data/sample efficiency** of your RL algorithm)
  * **robustness/sensitivity** (i.e., how the RL algorithm behaves if you change certain hyper-parameters); this is also important because RL algorithms can be very sensitive (from my experience)
  
  ## What is the difference between training and evaluation?
  
  During training, you want to find the policy. During the evaluation, you want to assess the quality of the learned policy (or RL algorithm). You can perform the evaluation even during training.
  
  > **Comments:**
  > **username_5**: [Here](https://groups.google.com/g/rl-list/c/7EHj4rtjY9I/m/ezLss7vV7wEJ) is another answer that is worth reading too.  
  
  ---
  
  ### Answer by username_2 on 2020-05-05. Score: 5
  
  # Reinforcement Learning Workflow
  
  The general workflow for using and applying reinforcement learning to solve a task is the following.
  
  [![enter image description here](https://i.sstatic.net/RV6cf.jpg)](https://i.sstatic.net/RV6cf.jpg)
  
  1. **Create the Environment**
  2. **Define the Reward**
  3. **Create the Agent**
  4. **Train and Validate the Agent**
  5. **Deploy the Policy**
  
  ---
  
  # Training
  
  * Training in Reinforcement learning employs a **system of rewards** and **penalties** to compel the computer to solve a problem by itself.
  * Human involvement is **limited** to **changing** the environment and **tweaking** the system of rewards and penalties.
  * As the computer **maximizes the reward**, it is prone to seeking unexpected ways of doing it.
  * Human involvement is focused on **preventing** it from **exploiting the system** and **motivating** the machine to perform the task in the way expected.
  * Reinforcement learning is useful when there is no **“proper way”** to perform a task, yet there are rules the model has to follow to perform its duties correctly.
  * Example: By tweaking and seeking the optimal policy for deep reinforcement learning, we built an agent that in just **20 minutes** reached a superhuman level in playing **Atari games**.
  * Similar algorithms, in principle, can be used to build AI for an **autonomous car.**
  
  [![Car](https://i.sstatic.net/lplhT.png)](https://i.sstatic.net/lplhT.png)
  
  # Testing
  
  * Debugging RL algorithms is very hard. Everything runs and you are not sure where the problem is.
  * To test if it worked well, if the trained agent is good at what it was trained for, **you take your trained model and apply it to the situation it is trained for**.
  * If it’s something like **chess** or **Go**, you could **benchmark** it against other engines (say stockfish for chess) or human players.
  * You can also define metrics for performance, ways of measuring the quality of the agent’s decisions.
  * In some settings (e.g a **Reinforcement Learning Pacman player**), the game score literally defines the target outcome, so you can just evaluate your model’s performance based on that metric.
  
  
  ---
  
  ### Answer by username_3 on 2020-05-05. Score: 1
  
  The goal of the reinforcement learning (RL) is to use data obtained via interaction with the environment to **solve the underlying Markov Decision Process** (MDP). "Solving the MDP" is tantamount to finding the optimal policy (with respect to the MDP's underlying dynamics which are usually assumed to be stationary).
  
  **Training** is the process of using data in order to find the optimal policy.
  **Testing** is the process of evaluating the (final) policy obtained by training.
  
  Note that, since we're generally testing the policy on the same MDP we used for training, the **distinction between the training dataset and the testing set** is no longer as important as it is the case with say supervised learning. Consequently, classical notions of **overfitting and generalization** should be approached from a different angle as well.
  
  
  ---
  
  ### Answer by username_4 on 2020-05-05. Score: 1
  
  If you want, you can do training and testing in RL.
  Exactly the same usage, training for building up a policy, and testing for evaluation.
  
  In supervised learning, if you use test data in training, it is like cheating. You cannot trust the evaluation. That's why we separate train and test data.
  
  The Objective of RL is a little different.
  RL trying to find the optimal policy. Since RL collects the information by doing, while the agent explores the environment (for more information), there might be a loss in the objective function. But, it might be inevitable for a better future gain.
  
  Multi-arm bandit example, If there are 10 slot machines. They will return random amounts of money. They have different expected returns. I want to find the best way to maximize my gain. easy, I have to find the machine with the greatest expected return and use only the machine. How to find the best machine?
  
  If we have a training and testing (periods),
  For example, I will give you an hour of the training period, so it doesn't matter if you lose or how much you earn. And in the testing period, I will evaluate your performance.
  
  What would you do? In the training period, you will try as much as possible, without considering the performance/gain. And in the testing period, you will use only the best machine you found.
  
  This is not a typical RL situation. RL is trying to find the best way, Learning by doing. All the results while doing are considered.
  
  Suppose that I tried all 10 machines once each. And, the No.3 machine gave me the most money. But I am not sure that it is the best machine, because all the machines provide a RANDOM amount. If I keep using the No.3 machine, it might be a good idea, because according to the information so far, it is the best machine. However, You might miss the better machine if you don't try other machines due to randomness. But if you try other machines, you might lose an opportunity to earn more money.
  What should I do? This is a well-known Exploration and Exploitation trade-off in RL.
  
  RL trying to maximize the gain including the gains right now and the gains in the future.
  In other words, the performance during training is also considered as its performance.
  That's why RL is not unsupervised nor supervised learning.
  
  However, in some situations, you might want to separate training and testing. RL is designed for an agent who interacts with the environment. However, in some cases, (for example), rather than having an interactive playground, you have data of interactions. The formulation would be a little different in this case.
  
  
  ---
  
  