Xi editor: A modern editor with a backend written in Rust
I&#x27;ve never constructed an editor -- but I have used one before ;) so I thought I&#x27;d make a few suggestions about the architectural considerations off the top of my head. I&#x27;m an Emacs user, but I&#x27;m very excited about NeoVim. It looks like your project is another really interesting one. I hope it takes off.<p>I notice that you&#x27;ve choosen Rust as the programming language for the editor, a fine choice. Performance appears to be a key factor in your choice of language. However, realize that performance isn&#x27;t the reason one should use Rust or C++ for the backend of an editor. Let&#x27;s go though some simple back of the envelope calculations. A realistic goal for editors is to keep ordinary command and keystroke latency under 100ms, this should appear to users as if operations are performed instantaniously. You&#x27;ve picked a more aggressive goal of a maximum latency of 16ms, so let&#x27;s go with that.<p>Today, a slow processor runs perhaps 5,000,000,000 instructions per second (speed of Raspberry Pi 2). By the time the editor is ready for general use, most programmer&#x27;s development systems will probably run something like 20 times faster or 100 Billion instructions per second. For this rough calculation I&#x27;ll use the speed of a Raspberry Pi 2. In 16ms that&#x27;s 8 million instructions.<p>What this means is that in the backend of the editor we have plenty of time to perform the editing operations no matter what language is used <i>as long as it is a straight code path through the editor</i>. The pedestrian operations of the editor, like reading a keystroke or inserting a letter into the text don&#x27;t require Rust or C++ or C. They can be done with practically any programming language, even a language that runs a hundred times slower than C++ like Python. It is the iterated operations where performance will be a concern: searching big files, complex regular expression searches, syntax highlighting after each keystroke, autocomplete, refactoring, the GUI and interfacing to the GUI.<p>I couldn&#x27;t agree more with your requirement that the editor shouldn&#x27;t hang or crash or lose work. Further, a modern editor is a big project and won&#x27;t gain traction without additional contributors. From a performance perspective my gut feeling is that any language that performs no worse than 10 times slower than C is a fine choice. For safety, either managed memory (with GC etc.) or an approach like Rust is essential to allow code to be contributed. The JVM languages, while being attractive in many ways carry the burden of a big infrastructure; Jetbrains has figured it out but even after all these years Eclipse feels so klunky--this make me rule out Java, Clojure and Scala. I think this leaves at least Rust, Go, OCaml, Common Lisp, Dart, and Swift. (I&#x27;d probably pick Go hoping to get more contributors.)<p>What about ropes, a clever and much more modern data structure for holding strings than ancient Emac&#x27;s gap buffer. Ropes do indead have better asymptotic performace for many of the <i>basic</i> string operations: delete an arbitrary character, insert a substring, etc. What about all of the other operations that you&#x27;d like your editor to do? Fast searching, backwards or forwards? Handling UTF-8? Doing regular expression matching. Ropes turn the text being edited into a tree of text segments so high speed searching is much more difficult. An abstraction layer over the data structure to make it appear to be indexed like an ordinary array will make some things run much slower, for example, Boyer-Moore string searches, regular expression searching (done for code highlighting), etc.<p>Ropes or other persistent data structures for sequences will work because they can present an abstract indexed access interface to the rest of the program, but you will always be dealing with it though this abstraction layer just when you&#x27;d rather not. Processors can move several bytes per instruction cycle so the gap buffer can close it&#x27;s gap while editing a 10MB file in less than a millisecond. This leaves the data in one contiguous block. So for simple things the gap buffer is much faster than rope structures because for most simple activity all of the additions and deletions are at the cursor, where the gap is located, and for complex things there is a 1ms overhead before the operation starts (to close the gap) but then things run basically full processor speed without the requirement to go though the abstraction layers. The places where I see this as likely to be important in modern editors is in the design of incremental backups, persistent text attributes, memory mapped access to underlying files, and coordination with asynchronous plugins working on the buffer. I&#x27;ll be curious to see how this works out for Xi.<p>Keystrokes and plugins. Emacs can end up running lisp code, through a pretty slow lisp interpreter, on every keystroke. It&#x27;s completely imperceptable. So performance isn&#x27;t a reason to treat keybindings differently than other plugins. The overhead of IPC to a plugin is measured in micro seconds.<p>While supporting Sublime Text keybindings is fine. The programmers most likely to be sensitive to keybindings will probably want either Vim or Emacs keybindings. A good way to support multiple keybindings will make the editor appeal to a much broader set of users. Plugins seem ideal for this. A key comes in, is processed by a plugin and a few elemental backend operations are invoked by the plugin in response to the key. This seems completely reasonable.<p>The real issue with plugins is direct access to the edit buffer if the plugins run in a separate process and address space. This is complex especially if the buffer is being accessed asynchronously. How is syntax highlighting done in a plugin for example. Where does the syntax highlighting actually get stored? Does the plugin have to be informed each time the user scrolls the screen or types a key (probably yes). What asynchronous commands does the plugin send back to the editor to update the display. Is there a separate plugin working on autocomplete? Does it also have to parse the source code just like the syntax highlighting plugin? Well those are things to figure out, but the keystroke bindings seem much easier to address in a plugin. Plugins provide a convienient place to address the semantic difference between a modal editing model (used extensively by Vim and to a degree by Emacs editors) and the underlying editor backends capabilities.