Crafting Interpreters
Somewhat tangentially to this discussion, I&#x27;ve been burned by DSLs at work.<p>Personal rant follows.  Apologies for being vague about the details, I&#x27;ve also changed some of them.<p>Our internal product has had 2 different DSLs written for it, one a graphical one written early in the 2000s, and another textual one written about 6 years ago.  Both of them were partially driven by the same guy, the first one as an implementer and the second one as a manager.   The second one was supposed to replace the first one. That first, graphical one, was disliked by many internal users, and was never actually fully completed.<p>Many internal apps were written in both those DSLs, and so far we have kept full backwards compatibility with both of them, including still occasionally fixing bugs in the interpreter. We have a vague plan to retire the first (graphical) one in a couple of years, once everyone has migrated to the second one.<p>I was against this second DSL from the beginning, and had a big fight against that manager guy who suggested it.  I wanted to use an existing language instead of writing our own, <i>again</i>.  However I was promptly shut out from the decision making process.  From what I could tell this was because a guy on another team convinced that manager that writing a new language is trivial, using a commercial parser generator we have the license to.<p>I might mention that the product has many other features and the language is just one of them.  It might be the most user-visible one, but there are many layers beneath it.<p>So that second language was built essentially by a commitee, without my direct involvement, with another team, which proved to be a long and painful process that took many hours.  And to my great chagrin I did end up partially implementing the transpiler for this language and implementing some of framework required to bolt it into the existing product.<p>This was sort of fun for a while but I was fully aware I was deep into NIH land.<p>The language itself was simple enough, but many features added around it caused a large amount of bugs.  Features included an in-app editor with full auto-complete, customized validations and warnings, automatic refactorings, a simple but performant on-line coverage engine, a primitive debugger, and so on.   The bugs were mostly in the interface between our code and the code coming from the second team, since implementation of those features was split between us and them.  Also a large number of bugs came from the effort to be fully backwards compatible with the older DSL language, which had some weird features due to its graphical nature, in order to allow conversion from the old language to the new.<p>The initial rollout of the language was met with some success, but it was mainly users who were relieved they don&#x27;t have to the first (graphical) DSL anymore.  Then users of course started asking for more features in this language.<p>However, just then, most of the team left.  For about a year I was the only one left and I was swamped with work.  Now we do have a team back, but it&#x27;s mostly new people, and each of them is already responsible for an existing feature other than this DSL.<p>As a result, this DSL has very little documentation, except a tutorial course and some bare-bones help.   Also, to cut corners, me and the team lead purposefully made some design decisions that impact performance, making our app very slow at times when users write lots of long functions in this DSL.<p>But now we&#x27;re stuck.  The current team lead is practically the only one who knows about this language and all the nitty gritty design issues in it.  And as a team lead they don&#x27;t have time to spend on it.<p>My team lead tries to be continue being an IC (individual contributor), and fix one of more severe of the performance problems on her own, but since they also has to function as a team lead, it took a long time, and also led to a series of very difficult bugs which we still deal with to this day.<p>I made some effort to wrote a post-transpiler phase which essentially bypasses some of the performance problems users have ran into, making her work somewhat redundant, but it&#x27;s not quite enough.<p>At the same time, there are requests from users and internal support to integrate &quot;proper&quot; scripting languages (python) or to expose APIs for our app.<p>So all this soured writing compilers &#x2F; interpreters, and DSLs in general, for me.  I used to love learning new languages and wrote my own complete and fully tested parser for some small languages, including the C&#x2F;C++ preprocessor.  But now I can&#x27;t bear thinking about doing it.<p>Once you have customers for this DSL, with demands and bug reports, and you hardly have the manpower to actually dedicate to improving it, it isn&#x27;t fun anymore.  Not to mention I was against this NIH thing from the beginning, and it pains me to keep supporting it.<p>Currently I&#x27;m thinking about integrating C# as a kind of scripting language, since we&#x27;re already a Microsoft shop, although that has been changing in the last decade.  TBH this was suggested years ago by the same guy who was responsible for the 2 DSLs, but I rejected it then due to performance issues.  Now with AOT compilation, Roslyn, and VSCode &#x2F; Monaco and its extensions, this might just be the correct time.  I&#x27;m not yet sure how exactly this will play out, but we&#x27;ll see.