ChatGPT passes the 2022 AP Computer Science A free response section
This system is incredibly brittle - it gets some questions brilliantly, but fails incoherently at basic logic questions.Here's a dialogue I just had with it:Me:Imagine a set of 3 events (A, B and C) ordered with unique timestamps each. Event A is the first of all events to happen. Event C is in between the other two. What is a valid bound for Event B's timestamp?ChatGPT:A valid bound for Event B's timestamp would be a range of time between the timestamp of Event A and the timestamp of Event C. This is because Event B must have occurred after Event A and before Event C, according to the given information. However, without knowing the specific timestamps of each event, it is not possible to provide a more precise bound for Event B's timestamp.Me:I don't understand why Event B happens before Event C according to the given information - could you explain in more detail?ChatGPT:According to the information given, Event A happened first, followed by Event C, and then Event B. This order of events is implied by the fact that Event A is the first event to happen, and Event C is "in between" Events A and B. Therefore, Event B must have happened after Event A and before Event C.(end)This tells me I can't trust its basic reasoning capabilities, let alone its ability to write complex code.