OpenAI's Foundry leaked pricing says a lot
&gt; This really should not be a surprise, because even the standard-issue ChatGPT can pass the Bar Exam<p>No, it can’t.<p>The two things that together have sometimes gotten misrepresented that way in “game of telephone”  presentations are:<p>(1) that when tested on the multiple choice component of the multistate bar exam (not the whole bar exam), it got passing grades in two subjects (evidence and torts), not the whole multiple choice section; which is very much <i>not</i> the same thing as being able to pass the exam, and 50.3% overall (better than chance, since its four choices per question, but also very much not passing.) <a href="https:&#x2F;&#x2F;papers.ssrn.com&#x2F;sol3&#x2F;papers.cfm?abstract_id=4314839" rel="nofollow">https:&#x2F;&#x2F;papers.ssrn.com&#x2F;sol3&#x2F;papers.cfm?abstract_id=4314839</a><p>(2) that a set of law professors gave it the exams for four individual law courses and it got an average of C+ scores on those exams, which are minimally-passing grades (but not on the bar exam.) <a href="https:&#x2F;&#x2F;www.reuters.com&#x2F;legal&#x2F;transactional&#x2F;chatgpt-passes-law-school-exams-despite-mediocre-performance-2023-01-25&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.reuters.com&#x2F;legal&#x2F;transactional&#x2F;chatgpt-passes-l...</a>