Answers.Body: <p>As far as I know, <strong>green AI</strong> and <strong>red AI</strong> are very recent terms and/or research areas, but their importance will or should definitely be more highlighted in the coming years (for obvious reasons).</p>
  <p>I don't know how many people are actively and directly researching this topic, but, in the past, I've already read <a href="https://dl.acm.org/doi/pdf/10.1145/3381831" rel="nofollow noreferrer">a research paper on the topic</a>, so a few people are already trying to at least raise awareness about these issues. Moreover, they also suggest that researchers report the number of floating-point operations (FLOPs) performed during the experiments, in addition to the usual performance metrics (such as the accuracy). I don't remember all the details of this paper, but I really recommend that you read it. <a href="https://arxiv.org/pdf/1906.02243.pdf" rel="nofollow noreferrer">This</a> another possibly useful paper that people interested in this topic might want to read.</p>
  <p>Although I'm currently not doing research on this topic, from an algorithmic perspective, improvements in the following areas will probably contribute to greener AI</p>
  <ul>
  <li>zero-, one- or few-shot learning</li>
  <li>transfer learning</li>
  <li>sample/data efficiency</li>
  <li>computational complexity</li>
  </ul>
  <p>If you can learn from fewer samples and with less computation, you will probably produce less emissions.</p>
  <p>So, I think that making AI greener is definitely possible, as it's possible to use an <a href="https://en.wikipedia.org/wiki/Electric_car" rel="nofollow noreferrer">electric car</a> rather than a petrol/diesel car, although more research on the topics mentioned above (and other topics) needs to be done to make it practically useful.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-18T13:39:28.110Z
Answers.Text: Zero-, or- or few-shot learning and transfer learning should be all based on expensive pre-training. So far as I know some pre-trained models cannot be employed for specific tasks, please refer to the [Reasoning fine-tuning performance](https://arxiv.org/pdf/2101.03961.pdf#page=18) part. The pre-trained models are too large to fine-tune.
Answers.UserDisplayName: Lerner Zhang
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-18T13:42:05.950Z
Answers.Text: @LernerZhang You may be right, but if you can re-use some models for different tasks, without training them from scratch, that's already a win. Humans don't learn everything from scratch, so I think that making AI behave/learn more as humans do may actually be very beneficial. I am not an expert in the topic, but, from what I remember having read around, humans or the human brains are a lot more efficient than computers.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-18T13:25:12.920Z
Answers.LastActivityDate: 2022-06-11T11:12:05.757Z
Answers.LastEditDate: 2022-06-11T11:12:05.757Z
Answers.UserDisplayName: nbro
Body: <p>The recent advances in machine learning were <a href="https://ai.stackexchange.com/q/13233/5351">mostly achieved by the hardware</a>, and the hardware <a href="https://www.technologyreview.com/2019/03/26/136381/the-next-ai-explosion-will-be-defined-by-the-chips-we-build-for-it/" rel="nofollow noreferrer">is said</a> to <a href="https://blog.deeplearning.ai/blog/the-batch-youtube-vs.-conspiracy-theorists-robots-that-think-ahead-gpu-cpu-the-future-transformers-retooled" rel="nofollow noreferrer">continue driving</a> the development of AI, but I was still shocked by <a href="https://twitter.com/draecomino/status/1362051082273386500?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E1362051082273386500%7Ctwgr%5E%7Ctwcon%5Es1_c10&amp;ref_url=https%3A%2F%2Fpublish.twitter.com%2F%3Fquery%3Dhttps3A2F2Ftwitter.com2Fdraecomino2Fstatus2F1362051082273386500widget%3DTweet" rel="nofollow noreferrer">this thread</a> which reads that the projected future cost for the largest model would be 1B dollars in 2025. And I learned that universities are suffering from an academic AI brain drain partly due to the <a href="https://qr.ae/pNEhry" rel="nofollow noreferrer">scarce hardware resources</a>.</p>
  <p><a href="https://i.sstatic.net/R1dyY.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/R1dyY.png" alt="enter image description here" /></a></p>
  <p>Some people proposed the so-called <a href="https://dl.acm.org/doi/fullHtml/10.1145/3381831" rel="nofollow noreferrer">Green AI</a> that encourages sustainable AI development but provides few constructive methods to prevent the trend.</p>
  <p>I wonder if the redder and redder AI would be in fact truly inevitable. It seems to me that all companies should build an expensive compute infrastructure to be competitive, but I think the investment would be very risky since most companies cannot get a higher return.</p>
  <p>But on the other hand, we human beings have evolved tens of millions of years or billions of years(life) on earth with <a href="https://en.wikipedia.org/wiki/Estimates_of_historical_world_population#:%7E:text=Recent%20estimates%20of%20the%20%22total,the%20order%20of%20100%20billion." rel="nofollow noreferrer">hundreds of billions of brains</a> that have ever lived on earth as a whole &quot;human brain&quot;. The biological wetware seems much much redder than the nowadays hardware and has consumed much much more energy than all the supercomputers. To make machines as intelligent as we humans shouldn't we pay as high a price? It reminds me of <a href="https://en.wikipedia.org/wiki/No_free_lunch_theorem" rel="nofollow noreferrer">the NFL theorem</a> but it should be imprecise in this scenario.</p>
  <p><em>So, will there be some promising techniques on the algorithm side that can make AI greener, affordable and sustainable in the future?</em> <strong>If not, could anyone please explain why AI should be unavoidably red and inevitably redder?</strong></p>
  
Tags: <machine-learning><deep-learning><ethics><green-ai>
Title: Will there be some promising techniques that can make AI greener and affordable in the future?
UserDisplayName: Lerner Zhang
ThreadText: # Will there be some promising techniques that can make AI greener and affordable in the future?
  
  (Asked by: username_0 on 2021-02-18)
  
  The recent advances in machine learning were [mostly achieved by the hardware](https://ai.stackexchange.com/q/13233/5351), and the hardware [is said](https://www.technologyreview.com/2019/03/26/136381/the-next-ai-explosion-will-be-defined-by-the-chips-we-build-for-it/) to [continue driving](https://blog.deeplearning.ai/blog/the-batch-youtube-vs.-conspiracy-theorists-robots-that-think-ahead-gpu-cpu-the-future-transformers-retooled) the development of AI, but I was still shocked by [this thread](https://twitter.com/draecomino/status/1362051082273386500?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E1362051082273386500%7Ctwgr%5E%7Ctwcon%5Es1_c10&ref_url=https%3A%2F%2Fpublish.twitter.com%2F%3Fquery%3Dhttps3A2F2Ftwitter.com2Fdraecomino2Fstatus2F1362051082273386500widget%3DTweet) which reads that the projected future cost for the largest model would be 1B dollars in 2025. And I learned that universities are suffering from an academic AI brain drain partly due to the [scarce hardware resources](https://qr.ae/pNEhry).
  
  [![enter image description here](https://i.sstatic.net/R1dyY.png)](https://i.sstatic.net/R1dyY.png)
  
  Some people proposed the so-called [Green AI](https://dl.acm.org/doi/fullHtml/10.1145/3381831) that encourages sustainable AI development but provides few constructive methods to prevent the trend.
  
  I wonder if the redder and redder AI would be in fact truly inevitable. It seems to me that all companies should build an expensive compute infrastructure to be competitive, but I think the investment would be very risky since most companies cannot get a higher return.
  
  But on the other hand, we human beings have evolved tens of millions of years or billions of years(life) on earth with [hundreds of billions of brains](https://en.wikipedia.org/wiki/Estimates_of_historical_world_population#:%7E:text=Recent%20estimates%20of%20the%20%22total,the%20order%20of%20100%20billion.) that have ever lived on earth as a whole "human brain". The biological wetware seems much much redder than the nowadays hardware and has consumed much much more energy than all the supercomputers. To make machines as intelligent as we humans shouldn't we pay as high a price? It reminds me of [the NFL theorem](https://en.wikipedia.org/wiki/No_free_lunch_theorem) but it should be imprecise in this scenario.
  
  *So, will there be some promising techniques on the algorithm side that can make AI greener, affordable and sustainable in the future?* **If not, could anyone please explain why AI should be unavoidably red and inevitably redder?**
  
  **Tags:** machine-learning, deep-learning, ethics, green-ai
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-02-18. Score: 3
  
  As far as I know, **green AI** and **red AI** are very recent terms and/or research areas, but their importance will or should definitely be more highlighted in the coming years (for obvious reasons).
  
  I don't know how many people are actively and directly researching this topic, but, in the past, I've already read [a research paper on the topic](https://dl.acm.org/doi/pdf/10.1145/3381831), so a few people are already trying to at least raise awareness about these issues. Moreover, they also suggest that researchers report the number of floating-point operations (FLOPs) performed during the experiments, in addition to the usual performance metrics (such as the accuracy). I don't remember all the details of this paper, but I really recommend that you read it. [This](https://arxiv.org/pdf/1906.02243.pdf) another possibly useful paper that people interested in this topic might want to read.
  
  Although I'm currently not doing research on this topic, from an algorithmic perspective, improvements in the following areas will probably contribute to greener AI
  
  * zero-, one- or few-shot learning
  * transfer learning
  * sample/data efficiency
  * computational complexity
  
  If you can learn from fewer samples and with less computation, you will probably produce less emissions.
  
  So, I think that making AI greener is definitely possible, as it's possible to use an [electric car](https://en.wikipedia.org/wiki/Electric_car) rather than a petrol/diesel car, although more research on the topics mentioned above (and other topics) needs to be done to make it practically useful.
  
  > **Comments:**
  > **username_0**: Zero-, or- or few-shot learning and transfer learning should be all based on expensive pre-training. So far as I know some pre-trained models cannot be employed for specific tasks, please refer to the [Reasoning fine-tuning performance](https://arxiv.org/pdf/2101.03961.pdf#page=18) part. The pre-trained models are too large to fine-tune.  
  > **username_1**: @username_0 You may be right, but if you can re-use some models for different tasks, without training them from scratch, that's already a win. Humans don't learn everything from scratch, so I think that making AI behave/learn more as humans do may actually be very beneficial. I am not an expert in the topic, but, from what I remember having read around, humans or the human brains are a lot more efficient than computers.  
  
  ---
  
  