Lisp: It's Not About Macros, It's About Read
Right now there is a divide among programmers.  One one side you have people like the author who crave the power of code-as-data more than they care about nice syntax and therefore love Lisp.  On the other side you have people who like more conventional syntax more than they care about code-as-data and therefore don't love Lisp.<p>Neither side can understand the other: one side says "why do you resist ultimate power?" and the other side says "how can you possibly think that your code is readable?"<p>My belief (and what I am starting to consider my life's work) is that the gap can be bridged.  Lisp's power comes from treating code as data.  But <i>all</i> code becomes data eventually; turning code into data is exactly what parsers do, and every language has a parser.  The author says "it's about read," but "read" (in his example) is just a parser.<p>The author asks "How would you do that in Python?"  The answer is that it would be something like this:<p><pre><code>  import ast
  
  class MyTransformer(ast.NodeTransformer):
    pass  # Implement transformation logic here.
  
  node = MyTransformer().visit(ast.parse("x = 1"))
  print ast.dump(node)
</code></pre>
This works alright, but what I'm after is a more universal solution.  With syntax trees there's a lot of support functionality you frequently want: a way to specify the schema of the tree, convenient serialization/deserialization, and ideally a solution that is not specific to any one programming language.<p>My answer to this question might surprise some people, but after spending a  lot of time thinking about this problem, I'm quite convinced of it.  The answer is Protocol Buffers.<p>It's true that Protocol Buffers were originally designed for network messaging, but they turn out to be an incredibly solid foundation on which to build general-purpose solutions for specifying and manipulating trees of strongly-typed data without being tied to any one programing language.  Just look at a system like <a href="http://scottmcpeak.com/elkhound/sources/ast/index.html" rel="nofollow">http://scottmcpeak.com/elkhound/sources/ast/index.html</a> that was specifically designed to store AST's and look how similar it is to .proto files.<p>(As an aside, programmers have spent the last 15 years or so attempting to use XML in this role of "generic language-independent tree structured serialization format," but it wasn't the right fit because most data is not markup.  Protocol Buffers can deliver on everything people <i>wanted</i> XML to be).<p>Why should manipulating syntax trees require us to <i>write</i> in syntax trees?  The answer is that it shouldn't, but this is non-obvious because of how inconvenient parsers currently are to use.  One of my life's goals is to help change that.  If you find this intriguing, please feel free to follow:<p><pre><code>  https://github.com/haberman/upb/wiki
  https://github.com/haberman/gazelle</code></pre>