Answers.Body: <p>I can't anwser your question on how much computing power you might need, but you'll need atleast a smallgrid to run the biggest model just looking at the memory requirments (175B parameters so 700GB of memory). The biggest gpu has 48 GB of vram <br/> I've read that gtp-3 will come in eigth sizes, 125M to 175B parameters. So depending upon which one you run you'll need more or less computing power and memory.<br/> (<a href="https://lambdalabs.com/blog/demystifying-gpt-3/" rel="nofollow noreferrer">https://lambdalabs.com/blog/demystifying-gpt-3/</a>) <br/>
  For an idea of the size of the smallest, &quot;The smallest GPT-3 model is roughly the size of BERT-Base and RoBERTa-Base.&quot;</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-05T21:35:00.387Z
Answers.LastActivityDate: 2020-08-08T14:26:39.677Z
Answers.LastEditDate: 2020-08-08T14:26:39.677Z
Answers.UserDisplayName: hal9000
Answers.Body: <p>I think it is premature to answer your question as OpenAI has not made GPT-3 available yet other than via a web-based API.  For more information see <a href="https://openai.com/blog/openai-api/" rel="nofollow noreferrer">OpenAI API</a>.</p>
  <p>From <a href="https://www.theverge.com/2020/6/11/21287966/openai-commercial-product-text-generation-gpt-3-api-customers" rel="nofollow noreferrer">OpenAI will start selling its text-generation tech, and the first customers include Reddit</a>, by James Vincent:</p>
  <p><strong>Access to the GPT-3 API is invitation-only, and pricing is undecided.</strong></p>
  <p>You can join the OpenAI wait list here: <a href="https://beta.openai.com/" rel="nofollow noreferrer">https://beta.openai.com/</a></p>
  <p>I read somewhere that to load GPT-3 for inferencing requires 300GB if using half-precision floating point (FP16).  There are no GPU cards today that even in a set of four will provide 300GB of video RAM.  For example, the best I believe you can do in a single desktop box is four NVLinked Nvidia RTX 8000 cards on a single motherboard.  Each card has 48GB of VRAM each.  That would only provide a total of 192GB of VRAM.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-09T15:56:44.757Z
Answers.LastActivityDate: 2020-08-09T16:30:45.497Z
Answers.LastEditDate: 2020-08-09T16:30:45.497Z
Answers.UserDisplayName: Brian O'Donnell
Body: <p>I know it cost around $4.3 million dollars to train, but how much computing power does it cost to run the finished program? IBM Watson chatbot AI only costs a few cents per chat message to use, OpeenAI Five seemed to run on a single gaming PC setup. So I'm wondering how much computing power does it need to run the finished ai program.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-06T11:09:57.540Z
Comments.Text: This is a programming/hardware issue, so it's off-topic here. Please, read https://ai.stackexchange.com/help/on-topic. This question is maybe more appropriate for Data Science SE.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-09T17:43:22.993Z
Comments.Text: Not a direct answer to the question but this is also interesting in this context: https://www.reddit.com/r/MachineLearning/comments/i49jf8/d_biggest_roadblock_in_making_gpt4_a_20_trillion/
Comments.UserDisplayName: present
Tags: <gpu><gpt>
Title: How much computing power does it cost to run GPT-3?
UserDisplayName: Simon Suh
ThreadText: # How much computing power does it cost to run GPT-3?
  
  (Asked by: username_0 on 2020-08-05)
  
  I know it cost around $4.3 million dollars to train, but how much computing power does it cost to run the finished program? IBM Watson chatbot AI only costs a few cents per chat message to use, OpeenAI Five seemed to run on a single gaming PC setup. So I'm wondering how much computing power does it need to run the finished ai program.
  
  **Tags:** gpu, gpt
  > **Comments:**
  > **username_1**: This is a programming/hardware issue, so it's off-topic here. Please, read https://ai.stackexchange.com/help/on-topic. This question is maybe more appropriate for Data Science SE.  
  > **username_2**: Not a direct answer to the question but this is also interesting in this context: https://www.reddit.com/r/MachineLearning/comments/i49jf8/d_biggest_roadblock_in_making_gpt4_a_20_trillion/  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2020-08-05. Score: 2 | Accepted answer
  
  I can't anwser your question on how much computing power you might need, but you'll need atleast a smallgrid to run the biggest model just looking at the memory requirments (175B parameters so 700GB of memory). The biggest gpu has 48 GB of vram   
   I've read that gtp-3 will come in eigth sizes, 125M to 175B parameters. So depending upon which one you run you'll need more or less computing power and memory.  
   (<https://lambdalabs.com/blog/demystifying-gpt-3/>)   
  For an idea of the size of the smallest, "The smallest GPT-3 model is roughly the size of BERT-Base and RoBERTa-Base."
  
  
  ---
  
  ### Answer by username_4 on 2020-08-09. Score: 1
  
  I think it is premature to answer your question as OpenAI has not made GPT-3 available yet other than via a web-based API. For more information see [OpenAI API](https://openai.com/blog/openai-api/).
  
  From [OpenAI will start selling its text-generation tech, and the first customers include Reddit](https://www.theverge.com/2020/6/11/21287966/openai-commercial-product-text-generation-gpt-3-api-customers), by James Vincent:
  
  **Access to the GPT-3 API is invitation-only, and pricing is undecided.**
  
  You can join the OpenAI wait list here: <https://beta.openai.com/>
  
  I read somewhere that to load GPT-3 for inferencing requires 300GB if using half-precision floating point (FP16). There are no GPU cards today that even in a set of four will provide 300GB of video RAM. For example, the best I believe you can do in a single desktop box is four NVLinked Nvidia RTX 8000 cards on a single motherboard. Each card has 48GB of VRAM each. That would only provide a total of 192GB of VRAM.
  
  
  ---
  
  