Ask HN: What was the best CS paper you read in 2017?
&gt;  TLA+ model-checker knocking out hard-to-test errors in protocols without having long process of formal verification qualifies it.<p>But that&#x27;s the mainstream of formal methods. 99% of formal methods in industry is using model checkers or static analyzers. What you call &quot;a long process of formal verification&quot;, meaning, I assume, formal manual proofs is almost nonexistent outside academia. So what you call &quot;lightweight&quot; formal methods is normally just called &quot;formal methods&quot;. What you call &quot;full&quot; formal methods is, for the moment at least, a research conjecture.<p>&gt; The model checkers usually only do partial verification due to state explosion.<p>That&#x27;s quite inaccurate. To be called a model checker, a model checker must check a model (in the formal logic sense) -- i.e. it must exhaustively search through the entire state space (some model checkers can exhaustively check infinite state spaces), as opposed to, say fuzzers or concolic tests. What happens in practice most of the time TLA+ is that the model checker exhaustively checks a finite-state specification (or a finite-state instance), which is derived (by the user) from an infinite-space one. Model checkers are not &quot;partial verification&quot;. They can be used to do partial verification. But so can proofs.<p>The <i>theoretical</i> difference between model checkers and proofs is that the latter is based on proof theory, while the former is based on model-theory. The challenge for model checkers is trying to <i>automatically</i> cover huge or infinite state-spaces; proofs have the same challenge, but they work manually with the <i>hope</i> that the human prover would be able to assist (this can actually be done with model-checkers as well, using refinements in TLA+, but TLC specifically is not that sophisticated). This hope is far from realized. <i>In practice</i>, most of the time, proofs hit a limit much sooner than model checkers, which is why they don&#x27;t scale as well <i>in practice</i>, and so are barely used (when they&#x27;re used, it&#x27;s usually to tie together results from model-checkers and&#x2F;or other automated tools).<p>&gt; The combo of ease of use with finite-space, model checking makes that form of TLA+ a lightweight, formal method.<p>Again, virtually all formal methods are like that, except those that are mostly used in academia. Manual formal proofs are not only not used because they are so prohibitively expensive, but because they&#x27;re simply not needed.<p>&gt; You contradicted the authors&#x27; own claim about its purpose.<p>That&#x27;s right. Those authors -- as accomplished as they are in their field -- have no experience in industrial software verification. Their claim is mere conjecture. I don&#x27;t know how someone is supposed to design a tool to serve a certain market if they&#x27;ve never researched that market. I find Lean to be very interesting (I&#x27;m playing with it as a hobby), but its authors make some unsubstantiated claims that are well outside their area of expertise. I remember their introductory paper sometimes says things like &quot;in practice&quot;, when it is clear to anyone who&#x27;s actually done that in practice that, at best, they could say &quot;in theory&quot;.<p>I should, however, qualify that Lean does have some features more oriented towards software verification, like -- most importantly -- integration with SMT solvers (which pretty much amounts to a model checker). But Lean does not differ in that regard from Why3 or Boogie, and I assume that&#x27;s not the part of Lean you were referring to.<p>&gt; I&#x27;ll be the first to agree each has a main goal of being designed by experts in such tools for use by veterans of or experts in such tools.<p>Those are not just veterans and experts, but mostly academics working on research projects. There has not been a single piece of large or even medium-sized software verified end-to-end (let alone &quot;fully&quot;) by anyone, ever, experts, veterans or otherwise. What you call &quot;full&quot; formal verification does not exist in any shape or form outside our hopes and dreams.<p>I think that the confusion between methods that are now purely research stage and are nowhere near ready for practical adoption, and actual working formal methods, does a lot of harm to formal methods. In general, I see a worrying trend of pure researchers, who have done little or no applied research, let alone actual industry work, presenting their pure research in a way that can be mistaken for &quot;nearly ready for primetime&quot; does a lot of harm for both research and industry. The field of formal methods in particular suffered a research winter (dried up funding) precisely because of such over-promises. Again, no one has <i>ever</i> &quot;fully&quot; verified a complex, large or even medium-sized real-world software, and even small, simplified software has virtually never been &quot;fully&quot; verified end-to-end outside academia. It&#x27;s fascinating research, but its no more than (pure!) research at this point.<p>In fact, I know a company that makes a commercial formal verification product (a very sophisticated sound static analyzer) that can also integrate manual proofs in Coq or Isabelle. One of their customers wanted to try writing Coq proofs, and the team -- all formal math academics -- actively tried to dissuade them (I assume they succeeded) from doing that. How? By showing them how it&#x27;s done.<p>&gt; That said, I&#x27;m definitely one of the people in favor of getting all that stuff or something similar in shape for better industry adoption. Alternatively or in parallel, attempting to stretch user-friendly methods like TLA+, Alloy, or Design-by-Contract to hand-prove whatever properties the others do.<p><i>All</i> formal methods designed for practical use are more-or-less user-friendly (or, at least aim to be), or they wouldn&#x27;t be used by anyone. &quot;All that stuff&quot; as in tools that are actually able to &quot;fully&quot; verify large, complex software cannot be made user friendly as it does not exist <i>at all</i>. I also don&#x27;t understand what you mean by &quot;stretching&quot; TLA+. I&#x27;ve used TLA+ extensively and have been learning Lean for quite some time, and there is <i>nothing</i> Lean can do when it comes to software verification that TLA+ can&#x27;t (although, there are different relative strengths). In fact things like refinement or specification&#x2F;verification of interactive&#x2F;concurrent&#x2F;distributed software -- all things that are of great importance in practice -- cannot be easily achieved in Coq&#x2F;Lean&#x2F;Isabelle without some serious work of embedding a logic like TLA in them (as, in fact, has been done when specifying&#x2F;verifying such software). If anything, I would say that <i>out of the box</i>, TLA+ is <i>more</i> powerful than Isabelle&#x2F;Coq&#x2F;Lean for a big portion of real-world software (as most of the software people write today is interactive&#x2F;concurrent&#x2F;distributed).<p>Which is why I don&#x27;t understand in what sense TLA+ is any &quot;lighter&quot; than Coq or Lean. The difference in power between Coq&#x2F;Lean and TLA+ has nothing to do with software, and everything to do with researching novel logics and formal mathematics.<p>I think that you are grossly misestimating what has actually been done in CompCert and other similar projects. Even the main author of CompCert readily acknowledges that model checkers scale better. A closer look at what they did would give you a better picture of the true state of affairs.