Body: <p>Imagine I have the following coins:</p>
  <pre><code>   coin_name success_prob failure_prob positive_reward negative_reward expected_reward
  1     coin_1    0.5229156   0.47708442       0.7901860      -1.0909964     -0.10729679
  2     coin_2    0.7211000   0.27889996       1.9919323      -1.6501755      0.97614858
  3     coin_3    0.8994624   0.10053758       1.0815446      -1.4799171      0.82402145
  4     coin_4    0.5609496   0.43905037       2.2192418      -2.5168569      0.13985590
  5     coin_5    0.7804740   0.21952601       2.3571215      -2.9998136      1.18113491
  6     coin_6    0.6032657   0.39673431       0.2093356      -0.2257176      0.03673507
  7     coin_7    0.5637658   0.43623417       1.3674532      -0.4321832      0.58239026
  8     coin_8    0.8766539   0.12334607       2.2418057      -1.1516021      1.82324220
  9     coin_9    0.9475227   0.05247732       1.4234440      -0.7954108      1.30700448
  10   coin_10    0.6872314   0.31276861       1.1958754      -3.4163135     -0.24667254
  11   coin_11    0.8325576   0.16744240       2.3964000      -1.8754937      1.68110388
  12   coin_12    0.5474203   0.45257967       1.1880020      -0.9212376      0.23340304
  13   coin_13    0.6919848   0.30801518       1.7261695      -2.7642130      0.34306353
  14   coin_14    0.6371918   0.36280818       1.4743202      -2.4181846      0.06208761
  15   coin_15    0.9073200   0.09267998       0.3470192      -2.7124723      0.06346562
  16   coin_16    0.7242582   0.27574183       2.2595799      -3.0144799      0.80530102
  17   coin_17    0.9050322   0.09496782       0.6906106      -2.0905425      0.42649051
  18   coin_18    0.9061948   0.09380525       0.2009429      -2.0933373     -0.01427263
  19   coin_19    0.8971712   0.10282884       0.8870097      -2.2459255      0.56485364
  20   coin_20    0.7199158   0.28008416       2.3908088      -2.9816879      0.88605758
  21   coin_21    0.8772376   0.12276242       2.2348944      -3.0280594      1.58880142
  22   coin_22    0.8146106   0.18538943       1.7627282      -2.7076841      0.93396098
  23   coin_23    0.8550912   0.14490880       1.6372164      -1.9157277      1.12236350
  24   coin_24    0.5003124   0.49968761       2.4862475      -2.5956930     -0.05313525
  25   coin_25    0.7376583   0.26234171       1.6736939      -0.5833858      1.08156777
  </code></pre>
  <p>Now, imagine a game where two players (player 1 always starts the game and both players start with reward = 0) take turns choosing and flipping coins while tallying their score (player 1 always starts) : the first player to reach a reward = + 10 wins and the first player to reach a reward = -10 loses.</p>
  <p>However now, imagine the following modification: At the start of the game - the success probabilities and rewards are unknown to both players. The players follow an &quot;exploration-exploitation paradigm&quot; where they try to estimate the probabilities and rewards of different coins based on the data they observe and the data of the other player - and then make decisions regarding what coin to pick based on the current state of the game.</p>
  <p><strong>My Question:</strong> I would like to learn how AI algorithms like Reinforcement Learning (e.g. Q-Learning, Monte Carlo Tree Search, etc.) can be used to develop strategies for optimally playing this game as more information about probabilities/rewards becomes known while the state of the game evolves (e.g. if a player is about to win, the other player might take a risk-averse decision and flip coins he knows have higher payoffs but lesser probabilities of success).</p>
  <p>Conceptually, I think it might be possible to design a random simulation in which both players play this game randomly again and again - and then try to incorporate different strategies based on Reinforcement Learning. I thought of two scenarios:</p>
  <ul>
  <li>Scenario 1: The probabilities and rewards stay the same in each game - in this scenario, perhaps the AI algorithm will eventually play enough in which success probabilities and rewards can be reliably estimated and an optimal policy can then be estimated</li>
  <li>Scenario 2: The probabilities and rewards are reset after each game - in this scenario, perhaps an AI algorithm can learn at what points is it better to explore vs stick to low risk strategies</li>
  </ul>
  <p><strong>I have never worked with Reinforcement Learning before - can someone please show me how Python libraries can be used to paly this game using Reinforcement Learning algorithms?</strong></p>
  <p>Thanks!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-04T08:45:41.487Z
Comments.Text: If I read correctly, your question is about *how can I implement this simulation* rather than *how AI algorithms learn*. If it is the case, you might want to implement a [custom Gym environment](https://stable-baselines3.readthedocs.io/en/master/guide/custom_env.html) that suits your scenario.
Comments.UserDisplayName: Minh-Long Luu
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-04T11:03:29.470Z
Comments.Text: Are players allowed to observe other player choices? Are they allowed to observe other player results?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-04T17:18:39.210Z
Comments.Text: One other thing: You will be generating the coin probabilities at the start, and the players will not know them. However, wil the players know the distribution of probabilties (e.g. will they know that each coin has `success_prob` chosen using equally probably from range $[0.5,0.95]$?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-04T18:05:10.143Z
Comments.Text: @ Neil Slater: Thank you for your reply! Players are allowed to observe other players choices and results. Yes, lets say that all coins have success_probabilities that are equally probable between some range (e.g. 0-5- 0.95). Thank you so much for everyting
Comments.UserDisplayName: stats_noob
Tags: <reinforcement-learning>
Title: Using AI to Play a Coin Flipping Game?
UserDisplayName: stats_noob
ThreadText: # Using AI to Play a Coin Flipping Game?
  
  (Asked by: username_0 on 2023-03-04)
  
  Imagine I have the following coins:
     coin_name success_prob failure_prob positive_reward negative_reward expected_reward
  1     coin_1    0.5229156   0.47708442       0.7901860      -1.0909964     -0.10729679
  2     coin_2    0.7211000   0.27889996       1.9919323      -1.6501755      0.97614858
  3     coin_3    0.8994624   0.10053758       1.0815446      -1.4799171      0.82402145
  4     coin_4    0.5609496   0.43905037       2.2192418      -2.5168569      0.13985590
  5     coin_5    0.7804740   0.21952601       2.3571215      -2.9998136      1.18113491
  6     coin_6    0.6032657   0.39673431       0.2093356      -0.2257176      0.03673507
  7     coin_7    0.5637658   0.43623417       1.3674532      -0.4321832      0.58239026
  8     coin_8    0.8766539   0.12334607       2.2418057      -1.1516021      1.82324220
  9     coin_9    0.9475227   0.05247732       1.4234440      -0.7954108      1.30700448
  10   coin_10    0.6872314   0.31276861       1.1958754      -3.4163135     -0.24667254
  11   coin_11    0.8325576   0.16744240       2.3964000      -1.8754937      1.68110388
  12   coin_12    0.5474203   0.45257967       1.1880020      -0.9212376      0.23340304
  13   coin_13    0.6919848   0.30801518       1.7261695      -2.7642130      0.34306353
  14   coin_14    0.6371918   0.36280818       1.4743202      -2.4181846      0.06208761
  15   coin_15    0.9073200   0.09267998       0.3470192      -2.7124723      0.06346562
  16   coin_16    0.7242582   0.27574183       2.2595799      -3.0144799      0.80530102
  17   coin_17    0.9050322   0.09496782       0.6906106      -2.0905425      0.42649051
  18   coin_18    0.9061948   0.09380525       0.2009429      -2.0933373     -0.01427263
  19   coin_19    0.8971712   0.10282884       0.8870097      -2.2459255      0.56485364
  20   coin_20    0.7199158   0.28008416       2.3908088      -2.9816879      0.88605758
  21   coin_21    0.8772376   0.12276242       2.2348944      -3.0280594      1.58880142
  22   coin_22    0.8146106   0.18538943       1.7627282      -2.7076841      0.93396098
  23   coin_23    0.8550912   0.14490880       1.6372164      -1.9157277      1.12236350
  24   coin_24    0.5003124   0.49968761       2.4862475      -2.5956930     -0.05313525
  25   coin_25    0.7376583   0.26234171       1.6736939      -0.5833858      1.08156777
  
  Now, imagine a game where two players (player 1 always starts the game and both players start with reward = 0) take turns choosing and flipping coins while tallying their score (player 1 always starts) : the first player to reach a reward = + 10 wins and the first player to reach a reward = -10 loses.
  However now, imagine the following modification: At the start of the game - the success probabilities and rewards are unknown to both players. The players follow an "exploration-exploitation paradigm" where they try to estimate the probabilities and rewards of different coins based on the data they observe and the data of the other player - and then make decisions regarding what coin to pick based on the current state of the game.
  My Question: I would like to learn how AI algorithms like Reinforcement Learning (e.g. Q-Learning, Monte Carlo Tree Search, etc.) can be used to develop strategies for optimally playing this game as more information about probabilities/rewards becomes known while the state of the game evolves (e.g. if a player is about to win, the other player might take a risk-averse decision and flip coins he knows have higher payoffs but lesser probabilities of success).
  Conceptually, I think it might be possible to design a random simulation in which both players play this game randomly again and again - and then try to incorporate different strategies based on Reinforcement Learning. I thought of two scenarios:
  
  Scenario 1: The probabilities and rewards stay the same in each game - in this scenario, perhaps the AI algorithm will eventually play enough in which success probabilities and rewards can be reliably estimated and an optimal policy can then be estimated
  Scenario 2: The probabilities and rewards are reset after each game - in this scenario, perhaps an AI algorithm can learn at what points is it better to explore vs stick to low risk strategies
  
  I have never worked with Reinforcement Learning before - can someone please show me how Python libraries can be used to paly this game using Reinforcement Learning algorithms?
  Thanks!
  
  **Tags:** reinforcement-learning
  > **Comments:**
  > **username_1**: If I read correctly, your question is about *how can I implement this simulation* rather than *how AI algorithms learn*. If it is the case, you might want to implement a [custom Gym environment](https://stable-baselines3.readthedocs.io/en/master/guide/custom_env.html) that suits your scenario.  
  > **username_2**: Are players allowed to observe other player choices? Are they allowed to observe other player results?  
  > **username_2**: One other thing: You will be generating the coin probabilities at the start, and the players will not know them. However, wil the players know the distribution of probabilties (e.g. will they know that each coin has `success_prob` chosen using equally probably from range $[0.5,0.95]$?  
  > **username_0**: @ Neil Slater: Thank you for your reply! Players are allowed to observe other players choices and results. Yes, lets say that all coins have success_probabilities that are equally probable between some range (e.g. 0-5- 0.95). Thank you so much for everyting  
  
  ---
  
  ## 0 answers
  
  