Answers.Body: <p>When a reinforcement learning problem is described as deterministic, that means the environment is deterministic. In turn that means:</p>
  <ul>
  <li>All rewards are single values (instead of distributions) and always occur in the same way for the same state/action pair.</li>
  <li>All state transitions are fixed, performing the action <span class="math-container">$a$</span> in state <span class="math-container">$s$</span> leads to the same state <span class="math-container">$s'$</span> each time.</li>
  </ul>
  <p>This determinism does not apply to the learning process or algorithm. Often these have some randomness to ensure exploration. It also says nothing about the agent's knowledge of the environment - a model-free agent starts with no knowledge of the environment and must discover the results.</p>
  <p>The results of e.g. moving to [1,2] and falling into a pit are deterministic in that this will always happen in the same way on repeated attempts. You can take advantage of this when designing the agent - if it discovers a pit on one episode, it will know it is there from then on.</p>
  <p>If you know an environment is deterministic, then you can take advantage of this knowledge. In simple environments you can set a high learning rate, because there are no probabilities to approximate.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-30T12:36:31.637Z
Answers.LastActivityDate: 2023-10-30T12:36:31.637Z
Answers.UserDisplayName: Neil Slater
Body: <p>I don't quite understand why the wumpus world problem is deterministic.</p>
  <p>I mean, if the agent starts at [1,1] and there is a pit at [1,2] then the agent cannot determine if there is a pit at [2,1] or [1,2]. He has to take a risk. Shouldn't the problem be non-deterministic?</p>
  
Tags: <knowledge-based-systems>
Title: Why is wumpus world problem deterministic?
UserDisplayName: Archaic
ThreadText: # Why is wumpus world problem deterministic?
  
  (Asked by: username_0 on 2023-10-30)
  
  I don't quite understand why the wumpus world problem is deterministic.
  
  I mean, if the agent starts at [1,1] and there is a pit at [1,2] then the agent cannot determine if there is a pit at [2,1] or [1,2]. He has to take a risk. Shouldn't the problem be non-deterministic?
  
  **Tags:** knowledge-based-systems
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-10-30. Score: 2 | Accepted answer
  
  When a reinforcement learning problem is described as deterministic, that means the environment is deterministic. In turn that means:
  
  * All rewards are single values (instead of distributions) and always occur in the same way for the same state/action pair.
  * All state transitions are fixed, performing the action $a$ in state $s$ leads to the same state $s'$ each time.
  
  This determinism does not apply to the learning process or algorithm. Often these have some randomness to ensure exploration. It also says nothing about the agent's knowledge of the environment - a model-free agent starts with no knowledge of the environment and must discover the results.
  
  The results of e.g. moving to [1,2] and falling into a pit are deterministic in that this will always happen in the same way on repeated attempts. You can take advantage of this when designing the agent - if it discovers a pit on one episode, it will know it is there from then on.
  
  If you know an environment is deterministic, then you can take advantage of this knowledge. In simple environments you can set a high learning rate, because there are no probabilities to approximate.
  
  
  ---
  
  