Hello HN!We're Oz and Einat, co-founders of lakeFS ( an open-source project that allows the creation of performant git-like repositories over an object store (i.e. S3).Prior to starting lakeFS we were VP of R&D and CTO at SimilarWeb, a (now-public) Israeli web analytics company whose business model is based on the collection and analysis of the internet's activity.
That equals A LOT of data. Our S3 data lake held nearly 10 Petabytes of data. [As a reminder, 1PB = 1 Million GBs]. At this scale, the workflows and technologies used by other data teams start to break down.Want to use a Data Warehouse like Snowflake to store and analyze the data? The bill from that could bankrupt the company.Want to duplicate one of your datasets temporarily to test a schema or feature change? When over a million S3 copy operations are involved, there's a significant non-zero chance some of those operations will fail.Your ability to perform analysis and innovate starts to slow considerably, which is fatal for an analytics org.Being able to do these operations without thought is something many analytics folk are able to take for granted… for now. But the trend is that data volumes will only keep growing (along with the expectations of what companies can do with their data).An elegant solution, we realized, to the obstacles above would be to introduce git-enabled workflows like those used in application development. 
Recovering from a pernicious error in a million S3 files shouldn't require a full day or even week of work to fix… instead let's make it an instantaneous revert operation to a previous commit.The challenge to implement this type of functionality is a technical one, one we took it upon ourselves to solve. It's been 1 year since the first public commit on lakeFS and we've now raised a $15M Series A to continue building and improving our vision.We've evolved a ton in the past year, completely refactoring the data model to remove dependency on Postgres. Fittingly, we now use rocksDB on the object store to persist the metadata lakeFS manages (with the added benefit of simplifying the installation process). Check out the roadmap to follow our progress on building out native integrations with other important technologies in the open data stack such as Spark, Hive Metastore, and Delta Lake.We encourage you to check out our Github repo: ( and documentation pages: ( proud of how far we've come, but know there's lots more work to do. Please do let us know your thoughts!