GPT-J-6B – A 6 billion parameter, autoregressive text generation model
I’m simultaneously surprised and unsurprised that announcements about Copilot get so much copyright discussion, while the GPT-like models don’t get nearly the same. Meanwhile, GPT-J is literally trained on pirated books (the books3 corpus is part of the Pile, which is the corpus this was trained on).<p>Charitably, it’s because licenses are already such a core discussion when github comes up.<p>Uncharitably, it’s because Copilot uses “our” community’s labor, while the GPTs use others’.