A Baby GPT
> This is a baby GPT with two tokens 0/1 and context length of 3, viewing it as a finite state markov chain.<p>What is actually the difference between this and a "pure" Markov chain? Is GPT simply a Markov chain?<p>EDIT: Better explained here: <a href="https://colab.research.google.com/drive/1SiF0KZJp75rUeetKOWqpsA8clmHP6jMg?usp=sharing" rel="nofollow">https://colab.research.google.com/drive/1SiF0KZJp75rUeetKOWq...</a>