Function calling and other API updates
GPT3.5 has been undergoing constant improvements, this price decrease (and context length increase) is great news!<p>The main problem I see with people using GPT3.5 is they try and ask it to "write a short story about aliens" and then they get back a crap boring response that sounds like it was written by an AI that was asleep at the wheel.<p>Good creative prompts are <i>long</i> and detailed, and to get the best results you really need to be able to tune temperature / top_p. Even small changes to a 3 paragraph prompt can result in a dramatic changes in the output, and unless people are willing to play around with prompting, they won't get good results.<p>None of the prompt guides I've seen really cover pushing GPT3.5 to its limit, I've published one of my more complicated prompts[1] but getting GPT3.5 to output good responses in just this limited sense has taken a lot of work.<p>As for the longer context, output length is different than following instructions, especially for a lot of use cases, pushing more input tokens is of as much interest as having more output tokens.<p>From what I have explored, even at 4k context length, with a detailed prompt earlier instructions in the prompt are "forgotten" (or maybe just ignored). The blog post calls out better understanding of input text, but again, I hope that isn't orthogonal to following instructions!<p>Finally in regards to function outputs, I wonder if it is a second layer they are running on top of the initial model output. I have always had a challenge getting the model to output parsable responses, there is a definite trade off between written creativity and well formatted responses, and to some extent having a creative AI extend out the format I specify has been really nice because it has allowed me to add features I did not think of myself!<p>[1] <a href="https://github.com/devlinb/arcadia/blob/main/backend/src/routes/promptstreamingws.ts#L133">https://github.com/devlinb/arcadia/blob/main/backend/src/rou...</a>