Xi-Editor Retrospective
Tangent&#x2F;clarification: the regex slowness issue is almost certainly due to using Perl-style exponential time regexes (which are required by Sublime syntax definitions) rather than &quot;regular languages&quot;.<p>The syntect library mentioned uses fancy-regex, which is explicitly different than Rust&#x27;s default regex crate, in that it supports regex constructs that require exponential backtracking:<p><a href="https:&#x2F;&#x2F;github.com&#x2F;fancy-regex&#x2F;fancy-regex" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;fancy-regex&#x2F;fancy-regex</a><p><i>In particular, it uses backtracking to implement &quot;fancy&quot; features such as look-around and backtracking, which are not supported in purely NFA-based implementations (exemplified by RE2, and implemented in Rust in the regex crate).</i><p>I think &quot;regexes&quot; have gotten bad name so I call them &quot;regular languages&quot; now.  That is, RE2, rust&#x2F;regex, and libc are &quot;regular language&quot; engines while Perl&#x2F;Python&#x2F;PCRE&#x2F;Java&#x2F;etc. are &quot;regex&quot; engines.<p>In my experience, lexing with regular languages can beat hand written code.  In the case of Oil vs. zsh, it&#x27;s beating hand-written C code by an order of magnitude (9-10x).<p><a href="http:&#x2F;&#x2F;www.oilshell.org&#x2F;blog&#x2F;2020&#x2F;01&#x2F;parser-benchmarks.html" rel="nofollow">http:&#x2F;&#x2F;www.oilshell.org&#x2F;blog&#x2F;2020&#x2F;01&#x2F;parser-benchmarks.html</a><p><a href="https:&#x2F;&#x2F;lobste.rs&#x2F;s&#x2F;77nu3d&#x2F;oil_s_parser_is_160x_200x_faster_than_it_was" rel="nofollow">https:&#x2F;&#x2F;lobste.rs&#x2F;s&#x2F;77nu3d&#x2F;oil_s_parser_is_160x_200x_faster_...</a><p>---<p><i>First, TextMate &#x2F; Sublime style syntax highlighting is not really all that great. It is quite slow, largely because it grinds through a lot of regular expressions with captures</i><p>pedantic: I think the issue is probably more lookahead than captures ?  You can do captures quickly in a &quot;regular language&quot; engine.<p><i>It may be surprising just how much slower regex-based highlighting is than fast parsers. The library that xi uses, syntect, is probably the fastest open source implementation in existence (the one in Sublime is faster but not open source). Even so, it is approximately 2500 times slower for parsing Markdown than pulldown-cmark.</i>