Answers.Body: <p>If I understand,</p>
  <ul>
  <li>With probability <span class="math-container">$p$</span> the robot select no re-roll (action <span class="math-container">$D$</span>).</li>
  <li>With probability <span class="math-container">$1-p$</span> the human uniformly select an action between <span class="math-container">$A,B,C,D$</span>.</li>
  </ul>
  <p>Thus the expected reward for if the robot helps is</p>
  <p><span class="math-container">$$R_{AH}= p\cdot D+(1-p)\frac{A+B+C+D}{4} = \frac{(A+B+C)(1-p)+D+3pD}{4}$$</span></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-24T20:51:13.750Z
Answers.LastActivityDate: 2021-02-24T20:51:13.750Z
Answers.UserDisplayName: Cohensius
Body: <p>I have this question that I'm kinda stuck on.</p>
  <p>It's a game scenario in which we set up an expectimax tree. In the game, you have 3 dice with sides 1-4 that you roll at the beginning. Then, depending on the roll, the player can choose one of the dice to reroll or not reroll anything. Points are assigned like so:</p>
  <ul>
  <li>10 points if there's 2 of a kind</li>
  <li>15 if there's 3 of a kind</li>
  <li>7 if there's a series like 1-2-3 or 2-3-4</li>
  <li>Otherwise, or if the sum is higher than the rewards from above, the score = sum of the rolls</li>
  </ul>
  <p>For additional context, this is an example expectimax tree I came up with, for the case that the player rolled a 1,2,4 and is considering rerolling or not:
  <a href="https://i.sstatic.net/gYR2T.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/gYR2T.png" alt="enter image description here" /></a></p>
  <p>Now lets introduce a new agent -- a robot that's supposed to help the human player. we assume</p>
  <ul>
  <li>the human player choses any action with uniform probability regardless of the initial roll</li>
  <li>there's a robot that, given a configuration of dice and the human's desired action, actually implements the action with probability 1-p and overrides it with a &quot;no reroll&quot; order with probability p&gt;0. It has no effect if the human's decision is already to not reroll.</li>
  </ul>
  <p>For that scenario, I came up with this expectimax tree:
  <a href="https://i.sstatic.net/6A2uI.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/6A2uI.png" alt="enter image description here" /></a></p>
  <p>Now for the part I'm actually stuck on -- lets define A, B, C, and D as the expected reward of performing actions &quot;reroll die 1&quot;, &quot;reroll die 2&quot;, &quot;reroll die 3&quot;, and &quot;no reroll.&quot; How do we find <span class="math-container">$R_H$</span>, the expected reward for the human acting without the robot's help, and <span class="math-container">$R_{AH}$</span> the expected reward for if the robot helps? We can't use p in the expression, we only have access to A,B,C,D and we're supposed to write it in the form <span class="math-container">$X + Y_p$</span></p>
  <p>*EDIT: I asked again and the question was worded weirdly. They said we should definitely use p. What was meant by not using p is X and Y themselves can't contain p. But Y will be multiplied by p in the final simplified form.</p>
  <p>For <span class="math-container">$R_{H}$</span> I think the answer should be <span class="math-container">$\frac{(A + B + C + D}{4}$</span> because of uniform distribution over A-D.</p>
  <p>I'm supposing that <span class="math-container">$R_{AH}$</span> would be <span class="math-container">$\frac{(A + B + C)(1-p) + D + Dp}{4}$</span>? Because the robot doesn't override with probability <span class="math-container">$1-p$</span>, but he can only override A-C and does with probability <span class="math-container">$p$</span>.</p>
  <p>I think something feels slightly wrong about my answer but I'm not sure what.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-24T07:29:30.660Z
Comments.Text: Are you sure you are not allowed to use p? The expected reward R_AH is a function of p.
Comments.UserDisplayName: Cohensius
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-24T13:34:14.763Z
Comments.Text: Hi. Could you please put your main **specific** question in the title? "Expectimax World Problem — Dice Rolling Game" is **not** a question, but the title of an article.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-24T17:06:08.710Z
Comments.Text: @Cohensius I asked the instructors and they clarified that you can use p. What was meant by not using p is X and Y themselves can't contain p. But Y will be multiplied by p in the final simplified form. It was a slight wording confusion. I added my best guess at a solution. However, I'm not 100% sure, and would appreciate confirmation :)
Comments.UserDisplayName: Manny
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-24T17:06:21.537Z
Comments.Text: @nbro done! sorry
Comments.UserDisplayName: Manny
Tags: <game-ai><game-theory><decision-trees><tree-search>
Title: Find the expected reward in an expectimax-based dice rolling game?
UserDisplayName: Manny
ThreadText: # Find the expected reward in an expectimax-based dice rolling game?
  
  (Asked by: username_0 on 2021-02-24)
  
  I have this question that I'm kinda stuck on.
  
  It's a game scenario in which we set up an expectimax tree. In the game, you have 3 dice with sides 1-4 that you roll at the beginning. Then, depending on the roll, the player can choose one of the dice to reroll or not reroll anything. Points are assigned like so:
  
  * 10 points if there's 2 of a kind
  * 15 if there's 3 of a kind
  * 7 if there's a series like 1-2-3 or 2-3-4
  * Otherwise, or if the sum is higher than the rewards from above, the score = sum of the rolls
  
  For additional context, this is an example expectimax tree I came up with, for the case that the player rolled a 1,2,4 and is considering rerolling or not:
  [![enter image description here](https://i.sstatic.net/gYR2T.png)](https://i.sstatic.net/gYR2T.png)
  
  Now lets introduce a new agent -- a robot that's supposed to help the human player. we assume
  
  * the human player choses any action with uniform probability regardless of the initial roll
  * there's a robot that, given a configuration of dice and the human's desired action, actually implements the action with probability 1-p and overrides it with a "no reroll" order with probability p>0. It has no effect if the human's decision is already to not reroll.
  
  For that scenario, I came up with this expectimax tree:
  [![enter image description here](https://i.sstatic.net/6A2uI.png)](https://i.sstatic.net/6A2uI.png)
  
  Now for the part I'm actually stuck on -- lets define A, B, C, and D as the expected reward of performing actions "reroll die 1", "reroll die 2", "reroll die 3", and "no reroll." How do we find $R\_H$, the expected reward for the human acting without the robot's help, and $R\_{AH}$ the expected reward for if the robot helps? We can't use p in the expression, we only have access to A,B,C,D and we're supposed to write it in the form $X + Y\_p$
  
  \*EDIT: I asked again and the question was worded weirdly. They said we should definitely use p. What was meant by not using p is X and Y themselves can't contain p. But Y will be multiplied by p in the final simplified form.
  
  For $R\_{H}$ I think the answer should be $\frac{(A + B + C + D}{4}$ because of uniform distribution over A-D.
  
  I'm supposing that $R\_{AH}$ would be $\frac{(A + B + C)(1-p) + D + Dp}{4}$? Because the robot doesn't override with probability $1-p$, but he can only override A-C and does with probability $p$.
  
  I think something feels slightly wrong about my answer but I'm not sure what.
  
  **Tags:** game-ai, game-theory, decision-trees, tree-search
  
  > **Comments:**
  > **username_1**: Are you sure you are not allowed to use p? The expected reward R_AH is a function of p.  
  > **username_2**: Hi. Could you please put your main **specific** question in the title? "Expectimax World Problem — Dice Rolling Game" is **not** a question, but the title of an article.  
  > **username_0**: @username_1 I asked the instructors and they clarified that you can use p. What was meant by not using p is X and Y themselves can't contain p. But Y will be multiplied by p in the final simplified form. It was a slight wording confusion. I added my best guess at a solution. However, I'm not 100% sure, and would appreciate confirmation :)  
  > **username_0**: @username_2 done! sorry  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-02-24. Score: 0 | Accepted answer
  
  If I understand,
  
  * With probability $p$ the robot select no re-roll (action $D$).
  * With probability $1-p$ the human uniformly select an action between $A,B,C,D$.
  
  Thus the expected reward for if the robot helps is
  
  $$R\_{AH}= p\cdot D+(1-p)\frac{A+B+C+D}{4} = \frac{(A+B+C)(1-p)+D+3pD}{4}$$
  
  
  ---
  
  