Markov Chains, Markov Process, Python, Discrete Time Markov, Probability.
variable We will now introduce a random variable X_t. The suffix t in X_t denotes the time step. At each time step t, X_t takes a value from the state space [1,2,3,…,n] as per some probability distribution. One possible sequence of values that X_t could take is {X_0=1, X_1=3, X_2=4, X_3=0,…,X_t=k}.