A weird epic ramble about Etsy homepages from the middle 2000’s
I’ve been sitting on a weird epic ramble about Etsy homepages from the middle 2000’s for the better part of 15 years now so I figured I would write it up as a thread 1&#x2F;<p>The story begins while I was still working in the financial industry in lower Manhattan, watching Visual Studio whitescreen for a half hour as it often did and questioning my life choices. 2&#x2F;<p>One day while that was happening a woman I knew DM’d me a link and said “I want everything on this website.” 3&#x2F;<p>I don’t know what she was trying to link me to because the site was broken, and it served me this error page. “Hey, sick error page,” I remember thinking. 4&#x2F;<p>This kept happening though, and eventually I came to be dimly aware that there was a new site called Etsy with a reputation for being full of beautiful stuff. 5&#x2F;<p>That reputation came from the homepage, which at the time was a grid of items that looked like this 6&#x2F;<p>The homepage grids changed periodically, and they were always interesting. The items would be beautiful individually, or beautiful together, or it’d contain some oblique joke. They’d be a rainbow of color, or they’d be the same shade of eggshell white, whatever. 7&#x2F;<p>It was this really wonderful handcrafted thing about handcrafted things, from the pre-personalization era. 8&#x2F;<p>Anyway I became familiar enough with this via osmosis to recognize the name Etsy when I saw it on the python.org jobs page, so I jumped and applied. 9&#x2F;<p>There were a few red flags in my Etsy interview which I was not conditioned to notice or emphasize at the time. It was scheduled for 3PM, but didn’t start until 3:30 because my interviewer was “really hung over.” 10&#x2F;<p>The interview was also about Lisp. There was this Lisp moment in the middle 2000s. You had to be there. 11&#x2F;<p>So after joining the company I eventually learned that the homepages were sourced from a feature called Treasury. Treasuries were a way for Etsy users to create collections, roughly the same as Pinterest these days. 12&#x2F;<p>Treasuries were selected and promoted to homepages via human cron. There was one poor employee living the Leonardo Da Vinci lifestyle, waking up every hour around the clock to turn over the homepage. 13&#x2F;<p>When I started she’d been at it for a while, and it took another six or eight months for that to end 14&#x2F;<p>Obviously this is not the kind of thing a human being should be asked to endure for a startup. But you have to understand, there was a problem of approximately this magnitude everywhere you looked. 15&#x2F;<p>-------
So the first version of Etsy (2005) was PHP talking to Postgres, written by someone learning PHP as he was doing it, which was all fine. Postgres was kind of an idiosyncratic choice for the time, but it didn’t matter yet and anyway that’s a different story.
-------<p>The p95 of search in that era was several minutes. The site was down for a full hour each week or so, and not on purpose. The human tragedy of someone waking up every hour to change the homepage felt kind of contained I guess 16&#x2F;<p>So like many of the frontend features of Etsy in those years, the Treasury was built in Flash by a visionary named Jared Tarbell, the “fourth founder.” He was more of a visual artist working in Flash as his medium than an engineer. 17&#x2F;<p>He lived in New Mexico and was very difficult to get ahold of. Once I talked to him and he told me he was working on etching intricate patterns into rocks with a laser, so he could drop them in the desert for future raccoon civilizations to wonder about. 18&#x2F;<p>Search ranking was a big problem for us, and we figured we could improve this if we knew things like “this or that listing is popular in popular treasuries.” 19&#x2F;<p>Beyond that, the homepage was the same for everyone looking at it. We thought maybe we could iterate away from that to something less artificially scarce. Getting an item on the homepage was a big deal for sellers, but the math of the situation made it unattainable for most. 20&#x2F;<p>The Flash architecture really held those ambitions back. There were only 250 Treasuries live on the site at any given time, which was a function of the RTMP server backend it was using. This was some kind of Flash chat server dingus. 21&#x2F;<p>We didn’t even have access to it. There was a contractor managing a Windows box someplace. And there was no database backing it, so as far as we knew once Treasuries were off the site they were lost forever. 22&#x2F;<p>In our heads this beautiful, wonderful thing feeding the public perception of Etsy was a small scale tragedy in terms of data loss and lost opportunity. 23&#x2F;<p>Back in the constant chaos of daily goings on on the engineering team, we were trying to get our arms around things like security vulnerabilities. We hired some pen testers for the first time, and they found problems everywhere. 24&#x2F;