Answers.Body: <p>I think the algorithm has changed minimally, but the necessary hardware has been trimmed to the bone.</p>
  
  <p>The number of gate transitions are reduced (perhaps float ops and precision too), as are the number of data move operations, thus saving both power and runtime.  Google suggests their TPU achieves a 10X cost saving to get the same work done.</p>
  
  <p><a href="https://cloudplatform.googleblog.com/2016/05/Google-supercharges-machine-learning-tasks-with-custom-chip.html" rel="nofollow">https://cloudplatform.googleblog.com/2016/05/Google-supercharges-machine-learning-tasks-with-custom-chip.html</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-23T19:35:43.730Z
Answers.LastActivityDate: 2016-08-23T19:35:43.730Z
Answers.UserDisplayName: Randy
Answers.Body: <p>Low precision enables high parallelism computation in Convo and FC layers.
  CPU &amp; GPU fixed architecture, but ASIC/FPGA can be designed based on neural network architecture </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-19T19:11:19.570Z
Answers.LastActivityDate: 2019-09-19T19:11:19.570Z
Answers.UserDisplayName: mahinlma
Answers.Body: <h2>Tensor operations</h2>
  
  <p>The major work in most ML applications is simply a set of (very large) tensor operations e.g. matrix multiplication. You can do <em>that</em> easily in an ASIC, and all the other algorithms can just run on top of that.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-27T01:10:08.820Z
Answers.Text: An important point is that the TPU uses 8 bit multiplication, which can be implemented much more efficiently than wider multiplication offered by the CPU. Such a low precision is sufficient and allows to pack many thousands of such multipliers on a single chip.
Answers.UserDisplayName: maaartinus
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-24T10:22:19.957Z
Answers.LastActivityDate: 2016-08-24T10:22:19.957Z
Answers.UserDisplayName: Peteris
Body: <p>We can read on <a href="https://en.wikipedia.org/wiki/TensorFlow#Tensor_processing_unit_.28TPU.29" rel="noreferrer">Wikipedia page</a> that Google built a custom ASIC chip for machine learning and tailored for TensorFlow which helps to accelerate AI.</p>
  
  <p>Since ASIC chips are specially customized for one particular use without the ability to change its circuit, there must be some fixed algorithm which is invoked.</p>
  
  <p>So how exactly does the acceleration of AI using ASIC chips work if its algorithm cannot be changed? Which part of it is exactly accelerating?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-08-17T12:00:40.703Z
Comments.Text: No mainstream AI technique that I'm aware of requires modification of the *algorithm*, though most are dependent on the ability to modify *data* (connection strengths, population members) etc.
Comments.UserDisplayName: NietzscheanAI
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-08-17T12:13:18.983Z
Comments.Text: So the only dynamic part like state of network is kept on some flash memory or drive?
Comments.UserDisplayName: kenorb
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-08-17T12:16:17.467Z
Comments.Text: According to https://en.wikipedia.org/wiki/Application-specific_integrated_circuit, modern ASICs can have RAM...
Comments.UserDisplayName: NietzscheanAI
Tags: <neural-networks><machine-learning><hardware><neuromorphic-engineering>
Title: How does using ASIC for the acceleration of AI work?
UserDisplayName: kenorb
ThreadText: # How does using ASIC for the acceleration of AI work?
  
  (Asked by: username_0 on 2016-08-17)
  
  We can read on [Wikipedia page](https://en.wikipedia.org/wiki/TensorFlow#Tensor_processing_unit_.28TPU.29) that Google built a custom ASIC chip for machine learning and tailored for TensorFlow which helps to accelerate AI.
  
  Since ASIC chips are specially customized for one particular use without the ability to change its circuit, there must be some fixed algorithm which is invoked.
  
  So how exactly does the acceleration of AI using ASIC chips work if its algorithm cannot be changed? Which part of it is exactly accelerating?
  
  **Tags:** neural-networks, machine-learning, hardware, neuromorphic-engineering
  
  > **Comments:**
  > **username_1**: No mainstream AI technique that I'm aware of requires modification of the *algorithm*, though most are dependent on the ability to modify *data* (connection strengths, population members) etc.  
  > **username_0**: So the only dynamic part like state of network is kept on some flash memory or drive?  
  > **username_1**: According to https://en.wikipedia.org/wiki/Application-specific_integrated_circuit, modern ASICs can have RAM...  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_4 on 2016-08-24. Score: 5 | Accepted answer
  
  ## Tensor operations
  
  The major work in most ML applications is simply a set of (very large) tensor operations e.g. matrix multiplication. You can do *that* easily in an ASIC, and all the other algorithms can just run on top of that.
  
  > **Comments:**
  > **username_5**: An important point is that the TPU uses 8 bit multiplication, which can be implemented much more efficiently than wider multiplication offered by the CPU. Such a low precision is sufficient and allows to pack many thousands of such multipliers on a single chip.  
  
  ---
  
  ### Answer by username_2 on 2016-08-23. Score: 3
  
  I think the algorithm has changed minimally, but the necessary hardware has been trimmed to the bone.
  
  The number of gate transitions are reduced (perhaps float ops and precision too), as are the number of data move operations, thus saving both power and runtime. Google suggests their TPU achieves a 10X cost saving to get the same work done.
  
  <https://cloudplatform.googleblog.com/2016/05/Google-supercharges-machine-learning-tasks-with-custom-chip.html>
  
  
  ---
  
  ### Answer by username_3 on 2019-09-19. Score: 0
  
  Low precision enables high parallelism computation in Convo and FC layers.
  CPU & GPU fixed architecture, but ASIC/FPGA can be designed based on neural network architecture
  
  
  ---
  
  