Deep Learning’s Diminishing Returns (2021)
None of these articles admit to a basic truth: deep learning is only a small slice of total computation, and even if it grew 2 orders of magnitude, would not be the largest consumer by area.<p>If DL costs are unsustainable, it means those other things (SAP, web hosting, and all the ridiculous stuff people waste cycles on) are even more unsustainable, and should be addressed first.