Answers.Body: <blockquote>
    <p>What's the difference between the two terms? Don't they mean the same thing?</p>
  </blockquote>
  
  <p>They mean different things, and can occur in any combination.</p>
  
  <h2>A known, deterministic environment</h2>
  
  <p>This is an environment where the researcher knows how to calculate all the transitions in advance of observing them, and the transition from state <span class="math-container">$s$</span> given action <span class="math-container">$a$</span> is always to the same next state <span class="math-container">$s'$</span> with the same reward <span class="math-container">$r$</span>.</p>
  
  <p>Example: Any classic board game against a fixed opponent (where any situation allows choice of action, the opponent always picks the same choice in the same situation)</p>
  
  <h2>An unknown, deterministic environment</h2>
  
  <p>This is an environment where the researcher does not have the knowledge to calculate all the transitions in advance of observing them, but any observation the transition from state <span class="math-container">$s$</span> given action <span class="math-container">$a$</span> is always to the same next state <span class="math-container">$s'$</span> with the same reward <span class="math-container">$r$</span>.</p>
  
  <p>Example: Simple mechanical physics environments, where initial measurements are unknown, imprecise or the researcher does not want to code knowledge of them into the agent. E.g. pole balancing.</p>
  
  <h2>A known, stochastic environment</h2>
  
  <p>This is an environment where the researcher knows all the rules about transitions, but those rules include transitions with random elements. Transition from state <span class="math-container">$s$</span> given action <span class="math-container">$a$</span> varies according to some probability function <span class="math-container">$p(s'|s,a)$</span> possibly so does the reward <span class="math-container">$p(r|s,a)$</span> - sometimes combined into joint probability function <span class="math-container">$p(r, s'|s,a)$</span>.</p>
  
  <p>Example: Any board game involving dice, e.g. Backgammon.</p>
  
  <h2>An unknown, stochastic environment</h2>
  
  <p>This is an environment where the researcher does not know all the rules, or can only calculate expected results with difficulty or to a low level of precision. Transition from state <span class="math-container">$s$</span> given action <span class="math-container">$a$</span> varies according to some <em>unknown</em> probability function <span class="math-container">$p(r, s'|s,a)$</span>. Learning the transition function may require many samples of <span class="math-container">$s, a$</span>, and this will be a statistical approximation.</p>
  
  <p>Example: In practice, most complex environments, including real-world physics with friction, fluids, non-perfect measurements.</p>
  
  <p>It is quite common for purpose of experiments to have a simulation where the environment is technically known (because the researcher wrote it, or has access to the code and underlying models), but that agents are written to treat it like this last, more challenging case. Agents that can figure out how to act without prior knowledge of the environment are often of interest.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-31T07:05:43.467Z
Answers.Text: so, for example - chess are deterministic, while games with probability are stochastic?
Answers.UserDisplayName: Piotr Żak
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-31T07:51:47.607Z
Answers.Text: @PiotrŻak: Yes. Backgammon is a classic board game that is stochastic, due to probilities introduced by the dice.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-24T11:42:03.687Z
Answers.LastActivityDate: 2020-01-24T11:42:03.687Z
Answers.UserDisplayName: Neil Slater
Body: <p>According to the book "Artificial Intelligence: A Modern Approach", "In a known environment, the outcomes (or outcome probabilities if the environment is stochastic) for all actions are given.", and in a deterministic environment, "the next state of the environment is completely determined by the current state and the action executed by the agent...". </p>
  
  <p>What's the difference between the two terms? Don't they mean the same thing?</p>
  
Tags: <reinforcement-learning><comparison><terminology><norvig-russell>
Title: What is the difference between the concepts "known environment" and "deterministic environment"?
UserDisplayName: Miguel A.
ThreadText: # What is the difference between the concepts "known environment" and "deterministic environment"?
  
  (Asked by: username_0 on 2020-01-24)
  
  According to the book "Artificial Intelligence: A Modern Approach", "In a known environment, the outcomes (or outcome probabilities if the environment is stochastic) for all actions are given.", and in a deterministic environment, "the next state of the environment is completely determined by the current state and the action executed by the agent...".
  
  What's the difference between the two terms? Don't they mean the same thing?
  
  **Tags:** reinforcement-learning, comparison, terminology, norvig-russell
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-01-24. Score: 2
  
  > What's the difference between the two terms? Don't they mean the same thing?
  
  They mean different things, and can occur in any combination.
  
  ## A known, deterministic environment
  
  This is an environment where the researcher knows how to calculate all the transitions in advance of observing them, and the transition from state $s$ given action $a$ is always to the same next state $s'$ with the same reward $r$.
  
  Example: Any classic board game against a fixed opponent (where any situation allows choice of action, the opponent always picks the same choice in the same situation)
  
  ## An unknown, deterministic environment
  
  This is an environment where the researcher does not have the knowledge to calculate all the transitions in advance of observing them, but any observation the transition from state $s$ given action $a$ is always to the same next state $s'$ with the same reward $r$.
  
  Example: Simple mechanical physics environments, where initial measurements are unknown, imprecise or the researcher does not want to code knowledge of them into the agent. E.g. pole balancing.
  
  ## A known, stochastic environment
  
  This is an environment where the researcher knows all the rules about transitions, but those rules include transitions with random elements. Transition from state $s$ given action $a$ varies according to some probability function $p(s'|s,a)$ possibly so does the reward $p(r|s,a)$ - sometimes combined into joint probability function $p(r, s'|s,a)$.
  
  Example: Any board game involving dice, e.g. Backgammon.
  
  ## An unknown, stochastic environment
  
  This is an environment where the researcher does not know all the rules, or can only calculate expected results with difficulty or to a low level of precision. Transition from state $s$ given action $a$ varies according to some *unknown* probability function $p(r, s'|s,a)$. Learning the transition function may require many samples of $s, a$, and this will be a statistical approximation.
  
  Example: In practice, most complex environments, including real-world physics with friction, fluids, non-perfect measurements.
  
  It is quite common for purpose of experiments to have a simulation where the environment is technically known (because the researcher wrote it, or has access to the code and underlying models), but that agents are written to treat it like this last, more challenging case. Agents that can figure out how to act without prior knowledge of the environment are often of interest.
  
  > **Comments:**
  > **username_2**: so, for example - chess are deterministic, while games with probability are stochastic?  
  > **username_1**: @PiotrŻak: Yes. Backgammon is a classic board game that is stochastic, due to probilities introduced by the dice.  
  
  ---
  
  