This is a classic, the folks at Sandia are really interesting to talk to about computer architecture, one of the few places where solid research continues.<p>The interesting thing for me when I got to Google was understanding how the entire Map &#x2F; Reduce paradigm worked. It really manages to greatly increase the amount of compute you can apply to a problem with a fairly painful restructuring. An interesting debate broke out about HPC (High Performance Computing) vs MPC (Massively Parallel Computing) and whether or not the architecture was irrelevant to the problem. Put specifically could a problem be expressed as an HPC problem, and then tranformed in some way to be expressed as an MPC problem. There is a pretty nice PhD in computer science in there somewhere, combining Amdahl&#x27;s law with the theory of computation.<p>Generally most folks started in the &quot;some things can&#x27;t be done by map reduce&quot; camp but over time more and more things have been shown to be doable that way, especially if you let the intermediate results float around inconsistently. I&#x27;ve always felt that combinators[1] had a role to play here. Sort of like the Z transform in signal theory, if you can decompose the problem into a set of combinator operations, is seems like you should be able to convert it from and HPC algorithm to a MPC algorithm. Just a conjecture as far as I can tell though.<p>[1] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Combinatory_logic#Combinatory_logic_in_computing" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Combinatory_logic#Combinatory_...</a>