Latency Exists, Cope (2007)
Latency exists, but no one cares about it until it crosses over a critical threshold!!<p>Throughput is the number most people care about. As long as latency remains &quot;below the critical rate&quot; (which is application dependent), throughput remains the more important figure.<p>As such: we can perform latency&#x2F;throughput tradeoffs in practice. Maybe even latency&#x2F;throughput&#x2F;simplicity tradeoffs.<p>* The absolute lowest latency is a single-thread system that blocks on everything. Wait until X is ready and immediately start doing Y. This happens to be very simple and elegant code in practice, but it can only do 1-thing at a time.<p>* However, people want more throughput. You can use pthreads, or golang &#x2F; fibers &#x2F; cooperative threads, to convert the &quot;single threaded&quot; code into higher-throughput code. (Get the CPU to &quot;work on something else&quot; while waiting for X). This makes latency worse, but increases throughput dramatically. Multi-core accelerates this pattern very naturally.<p>* For highest levels of throughput and lowest latency, you need an event-driven loop. Yes, the GetMessageW() loop in Win32, game-loops in video games, poll() in Linux, and the like. This is a bit difficult to use in practice, so people use async to help decompose the &quot;big loop&quot; in practice. Generalizing to multicore is difficult however. But virtually every &quot;high performance&quot; system I&#x27;ve ever seen comes down to some glorified event loop &#x2F; poll &#x2F; epoll &#x2F; async &#x2F; GetMessageW() pattern. Literally all of the ones I&#x27;ve ever seen.<p>-------<p>I&#x27;d say, work on throughput until latency becomes an issue. pthreads &#x2F; fibers are your #1 easiest tool IMO to reach for, and have adequate performance for ~100,000 events per second or so.