No.  Even in toy examples with little internal structure, consider the length of the sequence generated.  For both models we can include a synthetic &quot;end of sequence&quot; token which the model can produce when run generatively.  In the case of the Markov model the length distribution must be essentially geometric -- at each point that it&#x27;s possible to end the sequence, the model has some possibility of ending the sequence, conditioned only on the previous n characters and not on any other property.  The RNN states can model any arbitrary function, and thus are capable of generating sequences matching any arbitrary length distribution.