Functional, declarative audio applications
EDIT: I didn&#x27;t mean to sound so overconfident or showing off my credentials. This is my personal view given what I know, but there are things I don&#x27;t know (like good vs. bad OS audio APIs, how to tune kernels and pick good audio drivers, and whether you can get away with allocations, mutexes, or a JITted&#x2F;GC&#x27;d language on the audio thread). It&#x27;s possible that writing audio code in JS or non-real-time C++ won&#x27;t cause problems to users in practice, as long as it runs on a dedicated thread and doesn&#x27;t contend with the GUI for the event loop, and processes notes synchronously and deterministically, but I haven&#x27;t tried. For various opinions on this from people who have done more experimentation than me, see <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27128087" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27128087</a> and <a href="http:&#x2F;&#x2F;www.rossbencina.com&#x2F;code&#x2F;real-time-audio-programming-101-time-waits-for-nothing" rel="nofollow">http:&#x2F;&#x2F;www.rossbencina.com&#x2F;code&#x2F;real-time-audio-programming-...</a>.<p>----<p>I&#x27;ve spent many years working on computer music and DSP in general, and the last two years writing interactive audio apps like DAWs and trackers (traditionally written in real-time code).<p>Personally I feel that not only should the signal processing be handled in compiled code, but the process of allocating voices for incoming notes and sweeping filter parameters for existing notes (like a VST&#x2F;VSTi plugin) should be real-time&#x2F;allocation-free and deterministic.<p>Additionally for a sequencer or DAW-like program playing a scripted project file, the process of scanning through a document and determining the timing of events should also be real-time and deterministic, requiring it be synchronous with the code driving synthesis&#x2F;notes&#x2F;envelopes. (Some Web Audio music demos I&#x27;ve seen on HN, like the 64-step drum sequencer[1] or The Endless Acid Banger[2], are not deterministic. On a slow computer, when the browser&#x27;s JS engine stutters, notes stop being triggered even though existing notes sustain.) I think that requires that the &quot;document&quot; is stored in C&#x2F;C++&#x2F;Rust structs and containers, rather than GC&#x27;d objects like in JS.<p>(Processing user input depends on low latency without latency spikes rather than determinism and synchronous processing, but I don&#x27;t know if browsers are good at that either.)<p>At this point, I find it significantly easier to write a GUI in native toolkits like Qt, than to learn and write bindings for a GC&#x27;d language to access native data structures. And unfortunately there is a limited selection of mature native toolkits that are both not buggy (Qt is buggy) and has accessibility and internationalization, and optionally theming and native widgets. I still believe that writing a GUI in a native language <i>can</i> become a better user and developer experience than browsers, if more people invest effort and resources into better paradigms or implementations to fix native&#x27;s weaknesses compared to browsers (buggy libraries, no billion-dollar megacorps funding web browsers, apparently people nowadays think that React using a virtual DOM and recomputing parts of the UI that don&#x27;t change is a strength), while maintaining its strengths (less resources required, more predictable memory and CPU usage, and trivial binding to native code).<p>What&#x27;s the current state of native desktop GUIs? Qt is nearly good enough, but is run by a company trying to antagonize the open-source community and rip off commercial users, binds you to C++ (which is less pleasant than Rust), suffers from bugs (some neglected, deep-seated API design flaws), and handles reactivity poorly (though QProperty and transactions[3] promise to improve that aspect). GTK has cross-language support, but in gtk-rs, GTK&#x27;s refcounted design and inheritance and Rust&#x27;s move-based design and composition are fighting against each other. There&#x27;s other older APIs like WxWidgets, FLTK, FOX, etc, many of which I personally dislike as a user. Flutter is promising, but still buggy, the upper layers are virtual-DOM-based (I have reservations), and feels foreign on desktops (many missing keyboard shortcuts, one app (FluffyChat&#x27;s unfinished desktop port) is missing right-click menus, has broken keybinds, and has painfully slow animations).<p>Is there an alternative? Where would you draw the seam between a GC&#x27;d GUI and a realtime audio engine, to minimize the boilerplate glue code, and ensure that note processing and audio synthesis are real-time and deterministic (does this require being synchronous with each other?)?<p>I&#x27;ve seen a lot of software with a non-real-time or nondeterministic audio engine&#x2F;sequencer (even though I personally think it&#x27;s bad for users and unacceptable when I design a program). For example, ZorroTracker has an audio processing path written in JS (a RtAudio thread calling into V8 running in parallel with the GUI thread, calling JS bindings to native .dll&#x2F;.so audio generators), coupled with a (planned) sequencer written in JS and operating on JS objects, giving up on real-time. As I&#x27;ve mentioned, several browser-based audio projects I&#x27;ve seen on HN generate audio in Web Audio (real-time), but trigger notes asynchronously in non-real-time JS code.<p>&gt; Eventually it dawned on me that the task of composing audio processing blocks is itself free of those realtime performance constraints mentioned above: it&#x27;s only the actual rendering step that must meet such requirements.<p>Given my current understanding of real-time programming, I think that the task of feeding inputs into audio processing blocks is not free of realtime performance constraints. In any program with an audio sequencer, I&#x27;m not aware of how to get deterministic playback with predictable CPU runtime (which I do not think is worth compromising), without worrying about &quot;memory management, object lifetimes, thread safety, etc.&quot;, preallocating memory for synthesis, and picking a way to communicate with the UI (eg. lock-free queues for processing commands sequentially, and atomics or triple buffers for viewing the latest state). If you have a solution, do let me know!<p>[1]: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27112573" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27112573</a><p>[2]: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=26870666" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=26870666</a><p>[3]: <a href="https:&#x2F;&#x2F;doc-snapshots.qt.io&#x2F;qt6-dev&#x2F;template-typename-t-qproperty-t-proxy.html" rel="nofollow">https:&#x2F;&#x2F;doc-snapshots.qt.io&#x2F;qt6-dev&#x2F;template-typename-t-qpro...</a>