Reinventing the Network Stack for Compute-Intensive Applications
From the article -- <i>“The true bottleneck for processor throughput is the network interface used to connect a machine to an external network, such as an Ethernet, therefore severely limiting a processor’s data ingest capability,” said Dr. Jonathan Smith, a program manager in DARPA’s Information Innovation Office (I2O). “Today, network throughput on state-of-the-art technology is about 1014 bits per second (bps) and data is processed in aggregate at about 1014 bps. Current stacks deliver only about 1010 to 1011 bps application throughputs.”</i><p>I don&#x27;t disagree with this, but see the challenge not so much as the transmission speed between nodes, rather I see it as the &#x27;semantics&#x27; for expressing addressing and placement. (and yes, I am a big fan of RDMA :-))<p>One of the things I helped invent&#x2F;design back when I was at NetApp was some network attached memory or NAM. NetApp was among a number of people who realized that as memory sizes got larger, having them associated with a single node made less and less sense from a computational perspective.<p>One can imagine a &quot;node&quot; which lives in a 64 bit address space, has say 16GB of &quot;L3&quot; cache shared among anywhere from 12 to 12,000 instruction execution or &#x27;compute&#x27; elements. More general purpose compute would look more like a processor of today, more specialized compute looks like a tensor element or a GPU element.<p>&quot;RAM&quot; or generally accessible memory would reside in its own unit and could be &#x27;volatile&#x27; or &#x27;involatile&#x27; (backed by some form of storage). With attributes like access time, volatility, redundancy, etc.<p>That sort of architecture starts to look more like a super computer (albeit a non-uniform one) than your typical computer. With nodes &quot;booting&quot; by going through a self-test and then mapping their memory from the fabric and restarting from where they left off.<p>I thought I might get a chance to build that system at Google but realized that for the first time in my career that I was aware of I was prevented by my lack of a PhD. And what was worse, given the politics at the time, no one with a PhD would sign on as a &#x27;figurehead&#x27; (I reached out to a couple and we had some really great conversations around this) because of the way in which Google was evaluating performance at the time. (think grad school ++ but where only one author seems to get all the credit).<p>Now that I&#x27;ve had some free time I&#x27;ve built some of it in simulation to explore the trade-offs and the ability to predict performance based on the various channels. Slow going though.