OpenAI Releases Largest GPT-2 Text Generation Model
Tangent rant.<p>I'm skimming over some of the code at <a href="https://github.com/openai/gpt-2/blob/master/src/model.py" rel="nofollow">https://github.com/openai/gpt-2/blob/master/src/model.py</a> and I can't help but feel frustrated at how unreadable this stuff is.<p>1. Why is it acceptable to have single-letter variable names everywhere?<p>2. There's little to almost no documentation in the code itself. It's unclear what the parameters of any given function mean.<p>3. There are magic constants everywhere.<p>4. Function names are so terse... ("gelu", "attn")