Simulating History with ChatGPT
This is awesome. I&#x27;ve been speculating along similar lines, and it&#x27;s great to see this fleshed out.<p>I think &quot;correct the errors in this ChatGPT essay&quot; is a short-term viable homework exercise, but those errors might be gone in GPT-5 so I don&#x27;t think it&#x27;s long-term viable. Soon the LLM will just produce perfect essays at college level and there won&#x27;t be hallucinations for the student to correct.<p>However, the &quot;simulate the historical environment&quot; task is great and I think it has long-term potential. I think it can be taken further; rather than &quot;spot the errors that ChatGPT made&quot;, you could flip the script and make it &quot;survive 20 turns of conversation without making a historical error&quot;, so you&#x27;d need to know things like local traditions, perhaps the geography of the ancient settlement you&#x27;re studying, contemporaneous history like &quot;who is the emperor and what&#x27;s the sentiment towards him&quot; and so on.<p>I&#x27;m also envisioning that, since text-based exercises are extremely easy to game (just pipe your text prompt into ChatGPT), and since ChatGPT is soon going to be strictly superior to a high-school level student, we could get around this by having the homework as an in-person verbal role-play or Q&amp;A session, like a viva voce; essentially you have a verbal discussion with ChatGPT and you need to really know your material as it can dig into any part of the curriculum. Then ChatGPT can summarize each student&#x27;s interaction, and the teacher doesn&#x27;t have to sit through each individual one start-to-finish (1:1 exams are too time-consuming to be viable).<p>This round-trip through verbal interaction would potentially make the task more interesting (lots of people simply hate writing essays), shifts the focus away from tasks that will become obsolete (writing essays) in favor of ones that will be more relevant (human synthesis of ideas, and interpersonal interaction), and helps to mitigate the issue of LLM-assisted cheating by constructing an assignment that LLMs can&#x27;t trivially solve.