Laptop to Lambda: Outsourcing Everyday Jobs to Thousands of Transient Containers
Thank you for this nice writeup! This paper was led by my student Sadjad Fouladi (<a href="https:&#x2F;&#x2F;sadjad.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;sadjad.org&#x2F;</a>), part of a broader theme of coercing a &quot;purely functional-ish&quot; design onto everyday applications. There&#x27;s a less academic-ese version with a few extended results that was published in ;login: magazine (<a href="https:&#x2F;&#x2F;www.usenix.org&#x2F;system&#x2F;files&#x2F;login&#x2F;articles&#x2F;login_fall19_02_fouladi.pdf" rel="nofollow">https:&#x2F;&#x2F;www.usenix.org&#x2F;system&#x2F;files&#x2F;login&#x2F;articles&#x2F;login_fal...</a>). There was also a good analysis here (<a href="https:&#x2F;&#x2F;buttondown.email&#x2F;nelhage&#x2F;archive&#x2F;papers-i-love-gg&#x2F;" rel="nofollow">https:&#x2F;&#x2F;buttondown.email&#x2F;nelhage&#x2F;archive&#x2F;papers-i-love-gg&#x2F;</a>) and don&#x27;t miss <a href="https:&#x2F;&#x2F;buttondown.email&#x2F;nelhage&#x2F;archive&#x2F;http-pipelining-s3-and-gg&#x2F;" rel="nofollow">https:&#x2F;&#x2F;buttondown.email&#x2F;nelhage&#x2F;archive&#x2F;http-pipelining-s3-...</a> .<p>Some of Sadjad&#x27;s other work has included:<p>- ExCamera, which somewhat kicked off the trend of &quot;fire up 4,000 lambda workers in a burst, all working on one job&quot; -- for things like making a neural network search a video frame-by-frame, video compression in parallel at sub-GOP granularity, etc. (<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=16197253" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=16197253</a>)<p>- Salsify, which reused the &quot;purely functional&quot; video codec from ExCamera to improve WebRTC&#x2F;Zoom-style live video (<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=16964112" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=16964112</a> , <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=20794541" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=20794541</a>). Sadjad is giving an Applied Networking Research Prize talk about this work at IETF tomorrow.<p>- 3D ray-tracing (running PBRT on thousands of Lambdas, sending rays across the network), SMT&#x2F;SAT solving, etc.<p>We&#x27;re working to extend this line of work towards a more general, Wasm-based, &quot;purely functional&quot; operating system where most computations operate on content-addressed data and are content-addressed themselves, and determinism and reproducibility is properties guaranteed by the OS. Sort of analogous to how the operating systems of today (try to) guarantee memory isolation between processes. Imagine, e.g., a Git repository where you could represent the fact that &quot;blob &lt;x&gt; is the result of running computation &lt;y&gt; given tree &lt;z&gt; as input,&quot; and anybody can verify that result, or rebase the computation to run on top of their own input. If you&#x27;re interested in this general area, please consider doing a PhD at Stanford and&#x2F;or get in touch -- I&#x27;m hiring.