Answers.Body: <p>It's important that you puncture through the hype. What happens in AI / DL are fairly common algorithms. In particular, computationally it almost all comes down to matrix-matrix multiplication. And the TPU is a dedicated processor for doing matmatmult. That's all.</p>
  <p>So why? Well, I'm assuming you understand where matrix-matrix multiplication comes into the DL story.</p>
  <p>The matmatmult algorithm is very easy to code: it's literally 4 lines for a triply-nested loop. However, coding it that way gives you very low performance on modern processors. You can execute this much more efficiently, if you think a lot harder about it and code it more sophisticatedly; see <a href="https://dl.acm.org/doi/10.1145/1356052.1356053" rel="nofollow noreferrer">https://dl.acm.org/doi/10.1145/1356052.1356053</a></p>
  <p>From the extreme regularity of this operation it's also great for GPUs, which give you more bang for the buck, so they are popular in DL. And if you strip almost all instruction decoding out of your processor, and you hardwire the inner part of the matmatmult in a circuit, you get your TPU.</p>
  <p>So that's what it is: a dedicated matrix-matrix-multiplication processor. Not much intelligence to it, right?</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-11T13:45:02.733Z
Answers.LastActivityDate: 2023-02-11T17:50:41.227Z
Answers.LastEditDate: 2023-02-11T17:50:41.227Z
Answers.UserDisplayName: Victor Eijkhout
Body: <p>I have been learning a lot of low-level computer science concepts lately.</p>
  <p>I have heard google developing a TPU (Tensor Processing Unit) which is an &quot;AI Accelerator&quot;.</p>
  <p>This sounds interesting and I was wondering where should I start if I need to advance in one of these fields. Provided I have no knowledge of AI and ML.</p>
  <p>Should I buy some ML courses and learn all the math? How should I go about this?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-05-05T03:46:02.943Z
Comments.Text: My first introduction to AI since university was that I wanted to run GPT-2, and understand how it worked, so I went to the GPT-2 paper, and followed chains of citations backwards until I got to papers I could understand. That was a learning experience for the power of citations :) as for hardware, it probably depends which one you have
Comments.UserDisplayName: Criticize SE actions means ban
Tags: <artificial-intelligence><machine-learning>
Title: Where to start while learning about AI Hardware
ThreadText: # Where to start while learning about AI Hardware
  
  (Asked by: username_0 on 2023-02-10)
  
  I have been learning a lot of low-level computer science concepts lately.
  
  I have heard google developing a TPU (Tensor Processing Unit) which is an "AI Accelerator".
  
  This sounds interesting and I was wondering where should I start if I need to advance in one of these fields. Provided I have no knowledge of AI and ML.
  
  Should I buy some ML courses and learn all the math? How should I go about this?
  
  **Tags:** artificial-intelligence, machine-learning
  
  > **Comments:**
  > **username_1**: My first introduction to AI since university was that I wanted to run GPT-2, and understand how it worked, so I went to the GPT-2 paper, and followed chains of citations backwards until I got to papers I could understand. That was a learning experience for the power of citations :) as for hardware, it probably depends which one you have  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-02-11. Score: 2 | Accepted answer
  
  It's important that you puncture through the hype. What happens in AI / DL are fairly common algorithms. In particular, computationally it almost all comes down to matrix-matrix multiplication. And the TPU is a dedicated processor for doing matmatmult. That's all.
  
  So why? Well, I'm assuming you understand where matrix-matrix multiplication comes into the DL story.
  
  The matmatmult algorithm is very easy to code: it's literally 4 lines for a triply-nested loop. However, coding it that way gives you very low performance on modern processors. You can execute this much more efficiently, if you think a lot harder about it and code it more sophisticatedly; see <https://dl.acm.org/doi/10.1145/1356052.1356053>
  
  From the extreme regularity of this operation it's also great for GPUs, which give you more bang for the buck, so they are popular in DL. And if you strip almost all instruction decoding out of your processor, and you hardwire the inner part of the matmatmult in a circuit, you get your TPU.
  
  So that's what it is: a dedicated matrix-matrix-multiplication processor. Not much intelligence to it, right?
  
  
  ---
  
  