Scaling Etsy
As someone who scaled the database at a company that dealt with over a petabyte of data, here&#x27;s a few thoughts I have:<p>It seems like the biggest issue Etsy had was a lack of strong engineering leadership. They relied on an outside consultant to rewrite their application in a fundamentally flawed way. The lack of eng leadership resulted in both poor technical decisions being made and bad engineering processes for how the app was developed.<p>Etsy&#x27;s biggest technical mistake building an application using a framework that no one understood. This led to the predictable result that when they deployed the application to production, it didn&#x27;t work. Even if the application had worked, Etsy still would have needed to maintain the Twisted application indefinitely. Maintaining an application written in a framework no one understands sounds like a recipe for a disaster. Sooner or later you are going to run into issues that no one will know how to fix.<p>Process wise, Etsy made the mistake of not properly derisking the Twisted application. They only found out that it didn&#x27;t work when they were deploying it to production. They made the same issue a second time when they tried to deploy the replacement. When I&#x27;m building a new service, the first bit of code I write is to test that the fundamental design of the new service would work. Usually this only takes a few days instead of the months it takes to implement the full service. It sounds like Etsy could have setup a simple Twisted app that did a fraction of what the final version did. If so, they would have found a number of fundamental flaws in the design before having spent a ton of time building out the service.<p>To be honest, this story shows how a business can succeed even with a bad engineering team. It would be one thing if this sort of incident killed off Etsy. Instead Etsy has gone on to become the $5 Billion company it is today. I&#x27;m not saying engineering doesn&#x27;t matter. All I&#x27;m saying is you can build a pretty successful business even with a bad engineering team.