Answers.Body: <p>GPUs have orders of magnitude more computational power for AI workloads, so CPUs are practically never the bottleneck. TPUs are an alternative to GPUs, and you can estimate the number of required TPUs from the given TFLOPs.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-25T07:42:08.417Z
Answers.LastActivityDate: 2024-07-25T07:42:08.417Z
Answers.UserDisplayName: NikoNyrh
Body: <p>I often find studies estimating the number of GPUs required to deploy a deep learning model. <a href="https://arxiv.org/pdf/2104.04473" rel="nofollow noreferrer">This paper</a>, for example, estimates that ChatGPT would have required about 1024 GPUs in order to be trained over a month.</p>
  <p>Why do they only talk about the number of GPUs? Why do they not mention CPUs or TPUs (which presumably were used since it is a deep learning model)?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-24T09:54:47.620Z
Comments.Text: no, most likely they used nvidia gpus... apart from that, they don't use CPU time because it probably highly depends on many other factors, such as RAM, where instead GPU have their own memory (and CPU are substantially slower than GPU for DL...)
Comments.UserDisplayName: Alberto
Tags: <chatgpt><hardware>
Title: How many processors are required to run ChatGPT?
UserDisplayName: mirewine
ThreadText: # How many processors are required to run ChatGPT?
  
  (Asked by: username_0 on 2024-07-24)
  
  I often find studies estimating the number of GPUs required to deploy a deep learning model. [This paper](https://arxiv.org/pdf/2104.04473), for example, estimates that ChatGPT would have required about 1024 GPUs in order to be trained over a month.
  
  Why do they only talk about the number of GPUs? Why do they not mention CPUs or TPUs (which presumably were used since it is a deep learning model)?
  
  **Tags:** chatgpt, hardware
  
  > **Comments:**
  > **username_1**: no, most likely they used nvidia gpus... apart from that, they don't use CPU time because it probably highly depends on many other factors, such as RAM, where instead GPU have their own memory (and CPU are substantially slower than GPU for DL...)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-07-25. Score: 0
  
  GPUs have orders of magnitude more computational power for AI workloads, so CPUs are practically never the bottleneck. TPUs are an alternative to GPUs, and you can estimate the number of required TPUs from the given TFLOPs.
  
  
  ---
  
  