GPT-4
A class of problem that GPT-4 appears to still really struggle with is variants of common puzzles.  For example:<p>&gt;Suppose I have a cabbage, a goat and a lion, and I need to get them across a river.  I have a boat that can only carry myself and a single other item.  I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together.  How can I safely get all three across?<p>In my test, GPT-4 charged ahead with the standard solution of taking the goat first.  Even after I pointed this mistake out, it repeated exactly the same proposed plan. It&#x27;s not clear to me if the lesson here is that GPT&#x27;s reasoning capabilities are being masked by an incorrect prior (having memorized the standard version of this puzzle) or if the lesson is that GPT&#x27;S reasoning capabilities are always a bit of smoke and mirrors that passes off memorization for logic.