Answers.Body: <p>Transition probability matrix cannot be initialized. Your game world has some rules. Since we do not know these rules we can approximate them. To do this, we should run the game over and over and collect the outcome of each action.</p>
  
  <p>Here is a toy example: assume we control a plane. We want the plane to fly forward, but sometimes there may be wind in a random direction. So the plane won't end up in the desired position. Instead, it has some probability to fly left, right and forward. After the data is collected, we can estimate this probabilities.</p>
  
  <p>Another example is estimating n-gram model: assume we want to train a model that will suggest a word in search field. We would analyze a corpus and calculate all 2 word sequences (for <a href="https://en.wikipedia.org/wiki/Bigram" rel="nofollow noreferrer">bigram</a> model). Then, if a user starts typing <code>I</code> the model would suggest the word <code>am</code>, because <code>I am</code> occurs more frequently then <code>I did</code>, for example, and hence, the transition probability from <code>I</code> to <code>am</code> is greater.</p>
  
  <p>Here is good book on <a href="http://www.cs.columbia.edu/~mcollins/lm-spring2013.pdf" rel="nofollow noreferrer">Language Modeling</a></p>
  
  <p>But your question suggest me that you want something like reinforcement learning. Check out this <a href="https://www.youtube.com/watch?v=lfHX2hHRMVQ&amp;t=444s" rel="nofollow noreferrer">video</a> wich is a part of a great lectures series on RL by David Silver</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-02T23:25:55.173Z
Answers.LastActivityDate: 2020-03-02T23:58:48.203Z
Answers.LastEditDate: 2020-03-02T23:58:48.203Z
Answers.UserDisplayName: Aray Karjauv
Body: <p>Following on from <a href="https://ai.stackexchange.com/q/18332/2444">my other (answered) question</a>:</p>
  
  <p>With regards to the Markov process (chain), if an environment is a board game and its states are the various position the game pieces may be in, how would the transition probability matrix be initialised? How would it be (if it is?) updated?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-07-26T07:15:23.143Z
Comments.Text: Check out this fun project on [reddit](https://www.reddit.com/r/3Blue1Brown/comments/l3cckw/probability_distribution_in_monopoly_using_markov/?utm_medium=android_app&utm_source=share). It tries to calculate probability of arriving each square in the game of Monopoly.
Comments.UserDisplayName: Enes
Tags: <machine-learning><reinforcement-learning><game-ai><markov-chain>
Title: How is the probability transition matrix populated in the Markov process (chain) for a board game?
UserDisplayName: mason7663
ThreadText: # How is the probability transition matrix populated in the Markov process (chain) for a board game?
  
  (Asked by: username_0 on 2020-03-01)
  
  Following on from [my other (answered) question](https://ai.stackexchange.com/q/18332/2444):
  
  With regards to the Markov process (chain), if an environment is a board game and its states are the various position the game pieces may be in, how would the transition probability matrix be initialised? How would it be (if it is?) updated?
  
  **Tags:** machine-learning, reinforcement-learning, game-ai, markov-chain
  > **Comments:**
  > **username_1**: Check out this fun project on [reddit](https://www.reddit.com/r/3Blue1Brown/comments/l3cckw/probability_distribution_in_monopoly_using_markov/?utm_medium=android_app&utm_source=share). It tries to calculate probability of arriving each square in the game of Monopoly.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-03-02. Score: 1
  
  Transition probability matrix cannot be initialized. Your game world has some rules. Since we do not know these rules we can approximate them. To do this, we should run the game over and over and collect the outcome of each action.
  
  Here is a toy example: assume we control a plane. We want the plane to fly forward, but sometimes there may be wind in a random direction. So the plane won't end up in the desired position. Instead, it has some probability to fly left, right and forward. After the data is collected, we can estimate this probabilities.
  
  Another example is estimating n-gram model: assume we want to train a model that will suggest a word in search field. We would analyze a corpus and calculate all 2 word sequences (for [bigram](https://en.wikipedia.org/wiki/Bigram) model). Then, if a user starts typing `I` the model would suggest the word `am`, because `I am` occurs more frequently then `I did`, for example, and hence, the transition probability from `I` to `am` is greater.
  
  Here is good book on [Language Modeling](http://www.cs.columbia.edu/~mcollins/lm-spring2013.pdf)
  
  But your question suggest me that you want something like reinforcement learning. Check out this [video](https://www.youtube.com/watch?v=lfHX2hHRMVQ&t=444s) wich is a part of a great lectures series on RL by David Silver
  
  
  ---
  
  