What is GPT-3? written in layman's terms
I have some issues with how this article describes GPT and its impact. The scenarios that are described as using GPT-3 remains the stuff of science fiction as far as I know. They would require major technological breakthroughs in areas of autonomous agents, combining language models with 'concept-based' models, and more mundane things like labelling (can you imagine how difficult it would take to accurately label all legislation for 'nefarious details', given that even humans can't agree on what is nefarious?).<p>More specifically:<p>- training on data is a lossy process. In your examples, GPT would actually have a worse memory than your lawyer or therapist. There is no way to combine language models and something more abstract like 'facts'.<p>- GPT has shown zero ability to do anything consistently successfully without a human-in-the-loop. When it comes to bring AI models into production, this matters a lot. There's no way autonomous therapists are coming from GPT-3 when half the time the model spews out potentially dangerous garbage. You can't teach GPT-3 to not hurt people because it has no concept of people or hurting them. It JUST knows the shape of English.<p>- GPT is an unsupervised (in terms of data labelling work required) model. It has not made any breakthroughs in requiring labelled data for fine-tuning the model to do a specific task. Which remains a gigantic problem for productionalizing models. Like how are you going to build an autonomous therapist? That data remains as inaccessible and impossible to label as ever.<p>- Please stop telling people that neural nets are related to brain neurons. They have essentially no relationship other than the name and it just fosters this fear of Terminator and obscures the real issues that need to be thought about. This is just my personal opinion but I'm so tired of having to spend my time telling otherwise smart people who don't know better that we aren't close to Terminator.<p>GPT is an impressive technical accomplishment, but it's impact on the world has been exaggerated quite a bit IMO. Some of the demos I've seen are almost certainly smoke and mirrors or very carefully chosen, human-in-the-loop examples.