Facebook hit git performance issue on large repository
Facebook engineer here, working on this problem with Joshua.<p>What this comes down to is that git uses a lot of essentially O(n) data structures, and when n gets big, that can be painful.<p>A few examples:<p>* There's no secondary index from file or path name to commit hash.  This is what slows down operations like "git blame": they have to search every commit to see if it touched a file.<p>* Since git uses lstat to see if files have been changed, the sheer number of system calls on a large filesystem becomes an issue. If the dentry and inode caches aren't warm, you spend a ton of time waiting on disk I/O.<p>An inotify daemon could help, but it's not perfect: it needs a long time to warm up in the case of a reboot or crash. Also, inotify is an incredibly tricky interface to use efficiently and reliably. (I wrote the inotify support in Mercurial, FWIW.)<p>* The index is also a performance problem. On a big repo, it's 100MB+ in size (hence expensive to read), <i>and</i> the whole thing is rewritten from scratch any time it needs to be touched (e.g. a single file's stat entry goes stale).<p>None of these problems is insurmountable, but neither is any of them amenable to an easy solution. (And no, "split up the tree" is not an easy solution.)