How does GPT obtain its ability? Tracing emergent abilities of language models
So training on source code is what gave ChatGPT the ability to reason? And before that it was just doing text autocomplete? That seems pretty incredible.