The Computational Limits of Deep Learning
We can look Deep Learning's growing demands for computation and despair, or view those growing demands as an economic incentive to develop more powerful hardware that uses energy more efficiently at a lower marginal cost and in a more sustainable manner.In other words, Deep Learning's growing need for computing power seems to have reached a point at which it is now motivating fundamental research to find greener, cheaper, more energy-efficient hardware.The economic incentives are very powerful: Whichever companies (or organizations, or countries) find ways to harness the most computing power at the lowest marginal cost will win the race in this market.--PS. The same could be said for Bitcoin mining: it is also motivating fundamental research to develop greener, cheaper, more energy-efficient, more powerful hardware. Whoever finds ways to harness the most computing power at the lowest marginal cost will make the most money processing transactions on the network.