A Baby GPT
Finally, a concise explanation why this generation of chatbots –while absolutely superior to previous art (where applicable)— still throws me back to discovering markov chains*, 'cat'-ing countless amount of text into a 'corpora.txt' to build models of various personalities.<p>The idea that a larger corpora == better was quickly established, but I hit a ceiling quite soon on early RPi's.<p>It was possible to seed a start word for text generation, a Baby prompt we might call it?<p>Nice feature was being able to make a mini chain (model) of the user input sentence, then merge that statistics into the bots chain. The verbatim record of what had been said was obscured but some words and their relations —the experience— was kept. So I imagined, at least.<p>* I recommend giving the 'markovify' README a view: <a href="https://github.com/jsvine/markovify#basic-usage">https://github.com/jsvine/markovify#basic-usage</a><p>edit: LLMs still can't deliver an as convincing display of an Aleister Crowley-persona, but I can barely handle a prompt so we're even.