News about Mark Crispin (author of the original IMAP specification)
A few months ago, I started working on an IMAP server, and as part of that process I decided to read, as best I could, "the collected works of Mark Crispin". Of course this meant that I read through the latest IMAP specification (in its entirety, "cover to cover") but it also meant that I read through all of the old ones as well (if nothing else: Mark actually often stated one should).<p>"""It's instructive to read the IMAP3 document (RFC 1203), if only to see a dead-end branch in IMAP's evolution.""" -- <a href="http://mailman2.u.washington.edu/pipermail/imap-protocol/2007-October/000697.html" rel="nofollow">http://mailman2.u.washington.edu/pipermail/imap-protocol/200...</a><p>However, I honestly found this person fascinating: the more I read, the more I wanted to read; I thereby continued from the specifications, and have been reading everything I could get my hands on, scouring mailing lists old and new. I imagine that this is similar to how many might feel about their favorite author, only for me my favorite author is not Tolstoy, Dickens, or Shakespeare: it is Mark Crispin.<p>Obviously, like with most authors, I don't pretend to know anything about him as a person, but I look up to him as a writer. I thereby don't really know what I would say to him (nor even feel it terribly appropriate to do so at all); I do think, however, I can at least help some people here on Hacker News who might not know much about him appreciate what Mark Crispin has been doing for us in his life.<p>This man has been working, nearly constantly, on the IMAP protocol specification now for decades of his life; he has seen numerous challenges to compatibility and has had to make countless tradeoffs and compromises to both his vision for the protocol and his wording in specifications to keep making forward progress. Much of this is actually documented in years of mailing list archives.<p>"""This was a mistake.  We all acknowledge it to have been a mistake. However, the discussion about naming that took place in the early 1990s wasted at least 18 months of everybody's time (and probably reduced all of our lifespans by a few years due to high blood pressure).  What came up was a wretched compromise, but at least it let us do our work.""" -- <a href="http://mailman2.u.washington.edu/pipermail/imap-protocol/2006-April/000193.html" rel="nofollow">http://mailman2.u.washington.edu/pipermail/imap-protocol/200...</a><p>From all of this, I would like to say: I believe he was actually a visionary. Many people who use IMAP do not realize this, but Mark did not (from my reading) ever believe in the offline e-mail that Google and Microsoft are slowly obsoleting, even at the benign level of IMAP synchronization; in fact, his own client (alpine) doesn't even support that mode of operation: it is purely on "online" IMAP client with a tiny memory cache.<p>"""Email synchronization is a fool's errand; but there seem to be an abundant supply of fools that undertake it. Thus we have miserable mobile device email clients such as Mail.app on the iToy, BlackBerry, and the default Mail app on Android. At least Android has k9mail which - just barely - steps over the line into "usability".""" -- <a href="http://mailman2.u.washington.edu/pipermail/imap-protocol/2011-May/001402.html" rel="nofollow">http://mailman2.u.washington.edu/pipermail/imap-protocol/201...</a><p>If you go back to the early IMAP specifications, this is actually laid out in the rationale section: the argument is that in an age where users have too many devices to easily manage and network connectivity is nearly universal--or as I will call it, "Mark Crispin's 1988" (yes: <i>1988</i>, and this is already IMAP2)--it no longer makes sense to store e-mail on the client; he then lays out a strategy for an efficient mail-specific thin-client protocol, with everything from server-side search to server-side parsing.<p>"""Consequently, while the workstation may be viewed as an Internet host in the sense that it implements IP, it should not be viewed as the entity which contains the user's mailbox.  Rather, a mail server machine (sometimes called a "repository") should hold the mailbox, and the workstation (hereafter referred to as a "client") should access the mailbox via mail transactions.""" -- <a href="http://tools.ietf.org/html/rfc1064" rel="nofollow">http://tools.ietf.org/html/rfc1064</a><p>It is only, however, when one delves into the mailing lists where you truly get a sense for this: on various occasions, Mark has even looked at modern webmail systems as having more in common with IMAP than the alternatives people normally compare IMAP to (such as POP).<p>"""It's easy to dismiss all this, because only a few IMAP clients are sophisticated enough to take advantage of this state. The vast majority are glorified POP clients that babble IMAP protocol. This came about because of the long-obsolete notion that Internet access is a difficult and expensive commodity that requires that the client must keep a mirror of what's on the server. The success of webmail (which transforms the browser into the ultimate thin client) proves that this notion is complete nonsense today. Yet people persist in claiming it. Webmail won the war for the hearts and minds of users, not because webmail is so much better than IMAP, but rather because webmail is so much better than POP.""" -- <a href="http://mailman2.u.washington.edu/pipermail/imap-protocol/2009-December/001043.html" rel="nofollow">http://mailman2.u.washington.edu/pipermail/imap-protocol/200...</a><p>What struck me the most, though, is just how <i>often</i> people refused to see this: assuming that IMAP was something that it was not, or simply not giving Mark the respect he deserved from the history he has thinking about this problem; people oft would approach claiming they knew better, and wanted to start over. This meant that he often had to spend his time attempting to herd people towards a common goal, and defending what existed against misconceptions; even having to teach people what it meant to have a protocol at all.<p>"""Before assuming that you are smarter than the old guy, you ought to make sure that you really understand the problem.""" -- <a href="http://mailman2.u.washington.edu/pipermail/imap-protocol/2009-December/001045.html" rel="nofollow">http://mailman2.u.washington.edu/pipermail/imap-protocol/200...</a><p>He didn't just sit back and heckle, though: he provided long and detailed critiques; he imparted his knowledge to others, even as he saw people often ignore what he had learned. His explanations usually also gave you a history lesson, illuminating part of the process and showing not only why something works the way it does, but why it worked the way it did, and how that notion had to be stretched into what we are currently using today: you can learn a lot about not just IMAP, but protocols in general, from his writings.<p>"""Furthermore, if you design for the stupid, you must also design for the defiant.  If you fail to do that, you have learned absolutely nothing from my experience in the past 22 years.""" -- <a href="http://www.ietf.org/mail-archive/web/imap5/current/msg00005.html" rel="nofollow">http://www.ietf.org/mail-archive/web/imap5/current/msg00005....</a><p>There was a continual sobering undercurrent, however, with relation to how long it has taken IMAP to come to fruition (technically, it is still only a proposal). Hearing today's news brings back to mind one e-mail in particular from 2007, which I will now end this comment with (its a long one, but I consider it quite powerful, and in this context, I think it is important to include in its entirety).<p>"""<p>RFC 3501, like all human endeavors, is not perfect.  We have spent about 20 years in trying to get IMAP beyond Proposed Standard status.  We are probably going to fall back yet again with another Proposed Standard RFC for IMAP.<p>You can't assume that the specification is going to tell you everything that you need to know.  It will never happen.  We can address this particular question, but I can guarantee that someone will find another one after the publication of the new RFC.<p>Each IMAP specification update consumes a couple of years of my time. Invariably, there are months of "last calls" and inactivity, only to have someone call a "wait, we need to do this" at the last minute that pulls everything back.  Requests to review drafts don't work.<p>And, with the addition of more expository text to say (what is obvious to some people), we get a larger and more bloated document that people won't read.  There are already many IMAP implementations written by people who looked at the examples but never the formal syntax or the expository text, because their implementations blatantly violate both.<p>I understand -- and sympathize -- with the desire to remove reliance upon folklore and common sense.  I see no hope of that ever being accomplished.<p>The sad fact is that we are running out of time.  Given past history, there is little hope that it will reach full standard status under my tenure.<p>I don't think that it's a good use of the next decade or so to make a futile attempt to perfect the base specification.  It needs to be made good enough, and there needs to be general understanding of the architecture so that people don't blame their silly decisions on the base specification.<p>-- Mark --<p>""" -- <a href="http://mailman2.u.washington.edu/pipermail/imap-protocol/2007-May/000552.html" rel="nofollow">http://mailman2.u.washington.edu/pipermail/imap-protocol/200...</a>