OpenAI Releases Largest GPT-2 Text Generation Model
Tangent rant.I'm skimming over some of the code at https://github.com/openai/gpt-2/blob/master/src/model.py and I can't help but feel frustrated at how unreadable this stuff is.1. Why is it acceptable to have single-letter variable names everywhere?2. There's little to almost no documentation in the code itself. It's unclear what the parameters of any given function mean.3. There are magic constants everywhere.4. Function names are so terse... ("gelu", "attn")