Why Google stores billions of lines of code in a single repository (2016)
So as someone who previously worked for Google and now works for Facebook, it&#x27;s interesting to see the differences.<p>When people talk about Google&#x27;s monolithic repo they&#x27;re talking about Google3. This excludes ChromeOS, Chrome and Android, which ard all Git repos that have their own toolchains. Google3 here consists of several parts:<p>- The source code itself, which is essentially Perforce. This includes code in C++, Java, Python, Javascript, Objective-C, Go and a handful of other minor languages.<p>- SrcFS. This allows you to check out only part of the repo and depend on the rest via read-only links to what you need from the rest.<p>- Blaze. Much like Bazel. This is the system that defines how to build various artifacts. All dependencies are explicit, meaning you can create a true dependency graph for any piece of code. This is super-important because of...<p>- Forge. Caching of built artifacts. The hit-rate on this is very good and it consumes a huge amount of resources given the number of artifacts produced. Forge turns build times for some binaries from hours (even days) into minutes or even seconds.<p>- ObjFS. SrcFS is for source files. ObjFS is for built artifacts.<p>This all leads to what is usually a pretty good workflow like the ability to check out directories if you want to modify them and just use the read only version if you don&#x27;t. You can still step through the read only code with a debugger however.<p>Now Facebook I have less experience with (&lt;6 months) but broadly there are four repos: www, fbobjc, fbandroid and fbcode (C++, Java, Thrift services, etc). At one point these were Git but for various reasons ended up being migrated to Mercurial some years ago.<p>The FB case (IMHO) highlights just how useful it can be to have one repo. Google uses protobufs for platform independence. FB uses GraphQL at a client level and Thrift at the service level.<p>So one pain point is that, for example, you can modify a GraphQL endpoint in one repo but its used by clients in others (ie mobile clients). There are lots of warnings about making backward-incompatible changes, some of them excessively pessimistic because deterministically showing something will break some mobile build in another repo is hard.<p>Google3 has less of these problems because the code is in the same repo. On top of that, Google has spent a vast amount of effort making it so the same build and caching systems can handle C++ server code as well as Objective-C iOS app code. Basically if you&#x27;re working on Google3 you basically compile very little to nothing locally.<p>Engineers on Android, Chrome and ChromeOS however compile a lot of things locally and thus get far beefier workstations.<p>At FB the mobile build system doesn&#x27;t seem to be as advanced in that there is a far higher proportion of local building.<p>IIRC the Git people seemed to reject the idea of large code bases. Or, rather, their solution was to use Git submodules. There was (and maybe is?) parts of the Git codebase that didn&#x27;t scale because they were O(n). Apologies if I&#x27;m misspeaking here but I peripherally followed these discussions on HN and elsewhere years ago as someone from the outside looking in so I&#x27;m no authority on this.<p>The problem of course is that Git submodules don&#x27;t give you the benefits of a single repo and I&#x27;ve honestly not heard anyone say anything good about Git submodules.<p>Just to stress, the above is just my personal experience and I hope it&#x27;s taken as intended: general observations rather than complaints and definitely not arguing that one is objectively better than the other. There are simply tradeoffs.<p>Also, there are definite issues with Google3, like the dependency graph getting so large that even reading it in and figuring out what to build is a significant performance cost and optimization issue.