Dragonfly: Alibaba P2P file distribution system
Container image distribution seems to be one of the primary problems this tackles:<p>&quot;DevOps ... brings a lot of challenges: the efficiency of image distribution, especially when you have a lot of applications and require image distribution at the same time. Dragonfly works extremely well with both Docker and Pouch, and actually we are compatible with any other container technologies without any modifications of container engine.&quot;<p>FWIW, this was a similar problem that I tackled for Golang gopher gala hackathon 2015 - a custom bittorrent based docker image registry POC.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;netvarun&#x2F;docket" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;netvarun&#x2F;docket</a><p>Interestingly my problem statement was somewhat similar:<p>&quot;Large scale deploys are going to choke your docker registry. Imagine pulling&#x2F;deploying a 800mb base image (for example the official perl image) across 200 machines in one go. That&#x27;s 800*200 = 160GB [EDIT: Correction thanks to kingbirdy] of data that&#x27;s going to be deployed and it&#x27;ll definitely choke your private docker registry (and will take a while pulling in from the public image).&quot;