GPT Neo: open-source GPT model, with pretrained 1.3B & 2.7B weight models
Please test their models before you take it at face value.Eleuther has a history of claiming to replicate projects when they haven't. For example, they shipped a DALL-E repo a few days after OpenAI announced it (https://twitter.com/theshawwn/status/1348017515897659392) which was broken, and they've walked back their GPT-3 replication claims to replicating 1.5B due to the fact that their architecture doesn't scale.As far as I can tell, they're generating a large amount of hype with grandiose claims that they can't deliver on.All I care about is whether you like their models and actually use them in practice. If you do, please let me know and I'll pipe down. But so far, I haven't heard of anyone who uses anything they've produced, and that worries me. Has anyone?One specific claim they made: https://twitter.com/BlancheMinerva/status/134727697554780980..."DALL-E is quite straight forward and already coded. We just need data to train it."No, DALL-E is neither straightforward nor was it successfully coded, especially back on January 7th.Anyway, carry on. I really don't like speaking badly of AI projects, and I hope that they succeed. The model release today is a good step forward, assuming it works. But it might be better to have the expectation of "the models don't work" until proven otherwise.I'd also like to point out that there are some capable people doing work at Eleuther. Sid in particular is one of the best TPU hackers in the scene. I just wish they would scale down their claims, release more models, and not claim that they've done X until actually doing X. For example, the readme says they have "the ability to scale up to full GPT3 sizes (and possibly more!), using the mesh-tensorflow library," which they don't.