Answers.Body: <p>GPUs were ideal for AI boom because:</p>
  <ul>
  <li>They hit the right time</li>
  </ul>
  <p>AI has been researched for a LONG time. Almost half a century. However, that was all exploration of how algorithms would work and look. When NVIDIA saw that the AI is about to go mainstream, they looked at their GPUs and realized that the huge parallel processing power, with relative ease of programing, is ideal for the era that is to be. Many other people realized that too.</p>
  <ul>
  <li>GPUs are sort of general purpose accelerators</li>
  </ul>
  <p>GPGPU is a concept of using GPU parallel processing for general tasks. You can accelerate graphics, or make your algorithm utilize 1000s of cores available on GPU. That makes GPU awesome target for all kinds of use cases including AI. Given that they are already available and are not too hard to program, its ideal choice for accelerating AI algorithms.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-07T07:30:44.723Z
Answers.LastActivityDate: 2022-02-13T11:17:29.983Z
Answers.LastEditDate: 2022-02-13T11:17:29.983Z
Answers.UserDisplayName: Aleksandar Kostovic
Answers.Body: <p>Machine learning has been around since the 1960's.  They had computers that were less intelligent than the palm-pilots of the 1990's, and they did machine learning.</p>
  <p>What most folks call &quot;Machine Learning&quot; is deep neural networks like those that started getting competitive at vision-related tasks in the early 2010's (teens).  The gpgpu hardware standing on video cards (I had friends using the amiga bit-blitter for the task in the early 90's, @TadeuszWestawic) opened up a relative jump in performance, the software to drive that hardware like the methods from Hinton were mature and effective, and then some of the networks started doing amazing things.  Personally I consider Giraffe (Lai, 2015) and AlexNet (Krizhevsky, 2012) to have been watershed moments that motivated the business folks to put resources into the field.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-02-10T23:36:37.573Z
Answers.LastActivityDate: 2022-02-10T23:36:37.573Z
Answers.UserDisplayName: EngrStudent
Answers.Body: <blockquote>
  <p>The year 2012 is also generally considered the start of the “deep learning revolution”. The term
  “deep learning” refers to a branch of ML that is based on neural networks with many layers (hence the term “deep”). Although this basic technology had been around for many years, it was in 2012 when [KSH12] used deep neural networks (DNNs) to win the ImageNet image classification challenge by such a large margin that it caught the attention of the wider community. Related advances on other hard problems, such as speech recognition, appeared around the same time (see e.g., [Cir+10; Cir+11; Hin+12]). These breakthroughs were enabled by advances in <strong>hardware technology</strong> (in particular, the repurposing of fast graphics processing units (GPUs) from video games to ML), <strong>data collection technology</strong> (in particular, the use of crowd sourcing tools, such as Amazon’s Mechanical Turk platform, to collect large labeled datasets, such as ImageNet), as well as various <strong>new algorithmic ideas</strong>.</p>
  </blockquote>
  <p>Reference: <a href="https://probml.github.io/pml-book/book1.html" rel="nofollow noreferrer">Probabilistic Machine Learning: An Introduction</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-02-10T14:46:03.083Z
Answers.LastActivityDate: 2022-02-10T15:04:32.637Z
Answers.LastEditDate: 2022-02-10T15:04:32.637Z
Answers.UserDisplayName: Lerner Zhang
Body: <p>I listened to a talk attended by a panel consisting of two influential Chinese scientists: <a href="https://sites.google.com/site/gangwang6/" rel="nofollow noreferrer">Wang Gang</a> and <a href="https://www.crunchbase.com/person/kai-yu#section-overview" rel="nofollow noreferrer">Yu Kai</a>, among others.</p>
  <p>When asked about the biggest bottleneck in the development of artificial intelligence in the near future (3 to 5 years), Yu Kai, who has a background in the hardware industry, said that hardware would be the essential problem and that we should focus most of our attention on it. He gave us two examples:</p>
  <ol>
  <li>In the early development of computers, we compared machines by their chips;</li>
  <li>ML/DL, which has been very popular in recent years, would be almost impossible without the employment of Nvidia's GPUs.</li>
  </ol>
  <p>The fundamental algorithms existed already in the 1980s and 1990s, but AI went through 3 AI winters and was not empirical until we could train models with GPU boosted mega servers.</p>
  <p>Dr. Wang then commented on Yu's opinions, stating that we should also develop software systems because we cannot build an autonomous car even if we combine all the GPUs and computational power in the world.</p>
  <p>Then, as usual, my mind wandered off, and I started thinking: what if those who could operate supercomputers in the 1980s and 1990s had utilized the then-existing neural network algorithms and trained them with tons of scientific data? Some people at that time could obviously attempt to build the AI systems we are building now.</p>
  <p>But why did AI/ML/DL become a hot topic and practical only decades later? Is it merely a matter of hardware, software, and data?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-30T14:06:53.490Z
Comments.Text: Many people contributed to machine learning without Nvidia chips. For example Jacques Pitrat
Comments.UserDisplayName: Basile Starynkevitch
Tags: <machine-learning><deep-learning><history>
Title: Why did machine learning only become viable after Nvidia's chips were available?
UserDisplayName: Lerner Zhang
ThreadText: # Why did machine learning only become viable after Nvidia's chips were available?
  
  (Asked by: username_0 on 2019-07-07)
  
  I listened to a talk attended by a panel consisting of two influential Chinese scientists: [Wang Gang](https://sites.google.com/site/gangwang6/) and [Yu Kai](https://www.crunchbase.com/person/kai-yu#section-overview), among others.
  
  When asked about the biggest bottleneck in the development of artificial intelligence in the near future (3 to 5 years), Yu Kai, who has a background in the hardware industry, said that hardware would be the essential problem and that we should focus most of our attention on it. He gave us two examples:
  
  1. In the early development of computers, we compared machines by their chips;
  2. ML/DL, which has been very popular in recent years, would be almost impossible without the employment of Nvidia's GPUs.
  
  The fundamental algorithms existed already in the 1980s and 1990s, but AI went through 3 AI winters and was not empirical until we could train models with GPU boosted mega servers.
  
  Dr. Wang then commented on Yu's opinions, stating that we should also develop software systems because we cannot build an autonomous car even if we combine all the GPUs and computational power in the world.
  
  Then, as usual, my mind wandered off, and I started thinking: what if those who could operate supercomputers in the 1980s and 1990s had utilized the then-existing neural network algorithms and trained them with tons of scientific data? Some people at that time could obviously attempt to build the AI systems we are building now.
  
  But why did AI/ML/DL become a hot topic and practical only decades later? Is it merely a matter of hardware, software, and data?
  
  **Tags:** machine-learning, deep-learning, history
  
  > **Comments:**
  > **username_1**: Many people contributed to machine learning without Nvidia chips. For example Jacques Pitrat  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_2 on 2019-07-07. Score: 3
  
  GPUs were ideal for AI boom because:
  
  * They hit the right time
  
  AI has been researched for a LONG time. Almost half a century. However, that was all exploration of how algorithms would work and look. When NVIDIA saw that the AI is about to go mainstream, they looked at their GPUs and realized that the huge parallel processing power, with relative ease of programing, is ideal for the era that is to be. Many other people realized that too.
  
  * GPUs are sort of general purpose accelerators
  
  GPGPU is a concept of using GPU parallel processing for general tasks. You can accelerate graphics, or make your algorithm utilize 1000s of cores available on GPU. That makes GPU awesome target for all kinds of use cases including AI. Given that they are already available and are not too hard to program, its ideal choice for accelerating AI algorithms.
  
  
  ---
  
  ### Answer by username_3 on 2022-02-10. Score: 1
  
  Machine learning has been around since the 1960's. They had computers that were less intelligent than the palm-pilots of the 1990's, and they did machine learning.
  
  What most folks call "Machine Learning" is deep neural networks like those that started getting competitive at vision-related tasks in the early 2010's (teens). The gpgpu hardware standing on video cards (I had friends using the amiga bit-blitter for the task in the early 90's, @TadeuszWestawic) opened up a relative jump in performance, the software to drive that hardware like the methods from Hinton were mature and effective, and then some of the networks started doing amazing things. Personally I consider Giraffe (Lai, 2015) and AlexNet (Krizhevsky, 2012) to have been watershed moments that motivated the business folks to put resources into the field.
  
  
  ---
  
  ### Answer by username_0 on 2022-02-10. Score: 0
  
  > The year 2012 is also generally considered the start of the “deep learning revolution”. The term
  > “deep learning” refers to a branch of ML that is based on neural networks with many layers (hence the term “deep”). Although this basic technology had been around for many years, it was in 2012 when [KSH12] used deep neural networks (DNNs) to win the ImageNet image classification challenge by such a large margin that it caught the attention of the wider community. Related advances on other hard problems, such as speech recognition, appeared around the same time (see e.g., [Cir+10; Cir+11; Hin+12]). These breakthroughs were enabled by advances in **hardware technology** (in particular, the repurposing of fast graphics processing units (GPUs) from video games to ML), **data collection technology** (in particular, the use of crowd sourcing tools, such as Amazon’s Mechanical Turk platform, to collect large labeled datasets, such as ImageNet), as well as various **new algorithmic ideas**.
  
  Reference: [Probabilistic Machine Learning: An Introduction](https://probml.github.io/pml-book/book1.html)
  
  
  ---
  
  