The computers are fast, but you don't know it
Something all architecture astronauts deploying microservices on Kubernetes should try is benchmarking the latency of function calls.<p>E.g.: call a &quot;ping&quot; function that does no computation using different styles.<p>In-process function call.<p>In-process virtual (&quot;abstract&quot;) function.<p>Cross-process RPC call in the same operating system.<p>Cross-VM call on the same box (2 VMs on the same host).<p>Remote call across a network switch.<p>Remote call across a firewall and a load balancer.<p>Remote call across the above, but with HTTPS and JSON encoding.<p>Same as above, but across Availability Zones.<p>In my tests these scenarios have a performance range of about 1 million from the fastest to slowest. Languages like C++ and Rust will inline most local calls, but even when that&#x27;s not possible overhead is typically less than 10 CPU clocks, or about 3 nanoseconds. Remote calls in the typical case <i>start</i> at around 1.5 milliseconds and HTTPS+JSON and intermediate hops like firewalls or layer-7 load balancers can blow this out to 3+ milliseconds surprisingly easily.<p>To put it another way, a synchronous&#x2F;sequential stream of remote RPC calls in the <i>typical case</i> can only provide about 300-600 calls per second to a function that does <i>nothing</i>. Performance only goes downhill from here if the function does more work, or calls other remote functions.<p>Yet, every enterprise architecture you will ever see, without exception has layers and layers, hop upon hop, and everything is HTTPS and JSON as far as the eye can see.<p>I see K8s architectures growing side-cars, envoys, and proxies like mushrooms, <i>and then</i> having all of that go across external L7 proxies (&quot;ingress&quot;), multiple firewall hops, web application firewalls, etc...