DALL-E 2 open source implementation
This needs distributed training...<p>Years ago I made a shared tensor library[1] which should allow people to do training in a distributed fashion around the world.   Even with relatively slow internet connections, training should still make good use of all the compute available because the whole lot runs asynchronously with highly compressed and approximate updates to shared weights.<p>The end result is that every bit of computation added has <i>some</i> benefits.<p>Obviously for a real large scale effort, anti-cheat and anti-spam mechanisms would be needed to ensure nodes aren't deliberately sending bad data to hurt the group effort.<p>[1]: <a href="https://github.com/Hello1024/shared-tensor" rel="nofollow">https://github.com/Hello1024/shared-tensor</a>