LLMs are not greater than the sum of their parts: researchers
Apparently Chat GPT got a 163 on the LSAT, but I can&#x27;t get it to answer basic logic questions. E.g.<p>Prompt: If it rains today, I will drive home.  I did not drive home.  Did it rain today?<p>Output:<p>&gt; If the statement &quot;If it rains today, I will drive home&quot; is true and you did not drive home, it does not necessarily mean that it did not rain today. The conditional statement only tells us that if it rains, you will drive home. It does not tell us what you will do if it does not rain.<p>&gt; Therefore, it is possible that it rained today, but for some other reason, you did not drive home. Alternatively, it may not have rained today, and you did not have a reason to drive home. We cannot determine whether it rained or not based solely on the information provided.<p>Similarly, it can tell you what a converse is, but cannot give you the converse of a simple sentence:<p>&gt; The converse of the sentence &quot;If Anna goes to school today, she will eat lunch&quot; is:<p>&gt; &quot;If Anna eats lunch, then she went to school today.&quot;<p>&gt; The converse is formed by switching the order of the conditional statement and reversing the direction of the implication.<p>On the other hand, it&#x27;s quite good at math. It can easily solve simple two-variable algebraic equations.