GPT-2 is not as dangerous as OpenAI thought it might be
I think this is a pretty compelling claim. The published full GPT-2 samples are significantly better than the initial release, but pretty similar to the 774M corpus output. They&#x27;re disconcerting in as much as they&#x27;re a leap forward for AI writing, and they have remarkably strong tone consistency. But I see two major weaknesses with GPT-2 that make it hard to imagine effective malicious use.<p>First, the output is inconsistent. The scary-good examples are scattered among a lot of duds, and it semi-regularly breaks down into either general incoherence or distinctively robotic loops. You couldn&#x27;t turn it loose to populate a website or argue on forums without any human oversight, so it at most represents a cost reduction in the output of BS, not a qualitative shift.<p>Second, it&#x27;s absolutely crippled by length. Antecedents often go missing after about a paragraph. Nouns (especially proper nouns) that are likely once are likely in numerous roles, so stories will pull in figures like Obama as subject, object, and commentator on the same issue. Even stylistically, the tight guidelines of news ledes slacken within a few paragraphs, which results in a loss of focus and a rising likelihood of loops and gibberish.<p>GPT-2 is absolutely an impressive breakthrough in machine writing, I don&#x27;t mean to disparage that. But as far as it&#x27;s potential for deceit or trolling, it&#x27;s not particularly threatening. Quantity of output is rarely the limiting factor on impact for that, and GPT-2 doesn&#x27;t offer enough in sophisticated tone to make up for what it loses in basic coherence.<p>(For anyone wondering &quot;why is this sourced to a random tumblr?&quot;, nostalgebraist is some flavor of AI professional who&#x27;s played with GPT-2 to produce some pretty interesting results, and produced some other useful essays like an explanation of Google&#x27;s Transformer architecture (<a href="https:&#x2F;&#x2F;nostalgebraist.tumblr.com&#x2F;post&#x2F;185326092369&#x2F;the-transformer-explained" rel="nofollow">https:&#x2F;&#x2F;nostalgebraist.tumblr.com&#x2F;post&#x2F;185326092369&#x2F;the-tran...</a>).)