Answers.Body: <p>Some back of the envelope calculations :</p>
  
  <blockquote>
    <p>number of neurons in AI systems </p>
  </blockquote>
  
  <p>The number of neurons in AI systems is a little tricky to calculate, Neural Networks and Deep Learning are 2 current AI systems as you call them, specifics are hard to come by (If someone has them please share), but data on parameters do exist, parameters are more analogous to synapses (connections) than neurons (the nodes in between connections) somewhere in the range of 100-160 billion is the current upper number for specialized networks.</p>
  
  <p>Deriving the number of neurons in AI systems from this number is a stretch since these AIs emulate certain types of connections and sub assemblies of neurons, but let's continue...</p>
  
  <blockquote>
    <p>equal those of the human brain?</p>
  </blockquote>
  
  <p>So now let's look at the brain, and again this are all contested numbers. Number of neurons ~ 86 Billion, Number of Synapses ~ 150 Trillion, another generalization: average number of synapses per neuron ~ 1,744.</p>
  
  <p>So now we have something to compare, and I can't stress this enough, these are all wonky numbers, so let's make our life a little easier and divide :</p>
  
  <p>Number of Synapses (Brain ) : 150 trillion /  Number of parameters AIs : 150 billion = 1,000 or in other words current AIs would have to scale by a factor of one thousand their connections to be on par with the brain...</p>
  
  <p>Number of Neurons (Brain ) : 86 Billion / Number of Neurons AIs ( 150 billion / 1,744 )  = 86 Million equivalent AI Neurons</p>
  
  <p>Which makes sense, mathematically at least : you can multiply the factor ( 1000 ) times the current number of equivalent AI Neurons ( 86 million) to get the number of neurons in the human brain (86 Billion)</p>
  
  <blockquote>
    <p>When ?</p>
  </blockquote>
  
  <p>Well,let's use  moore's law ( number of transistors processing power doubles about every 2 years ) as a rough measure of technological progress: </p>
  
  <pre><code>     #AI NEURONS            YEAR
       86,000,000             2016
       172,000,000            2018
       344,000,000            2020
       688,000,000            2022
       1,376,000,000          2024
       2,752,000,000          2026
       5,504,000,000          2028
       11,008,000,000         2030
       22,016,000,000         2032
       44,032,000,000         2034
       88,064,000,000         2036
  
  
       # NEURONS HUMAN BRAIN 
       86,000,000,000
  </code></pre>
  
  <p>So, if all this made sense to you, somewhere around the year 2035. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-02-22T12:18:18.347Z
Answers.Text: https://arxiv.org/abs/1701.06538 , this might incline you revise your estimate. ;-) They want to scale up to a trillion parameters and I wouldn't be surprised if they did it this year, which would be a jump to 2022 on your timeline.
Answers.UserDisplayName: BlindKungFuMaster
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-25T19:16:57.987Z
Answers.Text: is the number of parameters in this analogy equivalent to the number of synapses?
Answers.UserDisplayName: committedandroider
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-26T20:18:11.367Z
Answers.Text: @committedandroider: roughly but yes, parameters here are weights in between neurons/nodes in ANN, in natural NN the equivalent is synaptic weights, which map to synapses.
Answers.UserDisplayName: Keno
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-14T08:02:28.057Z
Answers.LastActivityDate: 2016-11-22T19:43:08.360Z
Answers.LastEditDate: 2016-11-22T19:43:08.360Z
Answers.UserDisplayName: Keno
Answers.Body: <p>The answers so far haven't answered the question numerically, so here is my attempt to steer them in the direction I was seeking:</p>
  
  <p>The freely available <a href="http://www.deeplearningbook.org" rel="nofollow noreferrer">Deep Learning Book</a> has the following figure on page 27:</p>
  
  <p><a href="https://i.sstatic.net/iz2C4.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/iz2C4.png" alt="Size of neural nets over time"></a></p>
  
  <p>I question the blue fit line, as it seems that data points may be better described by a parabolic or exponential function. </p>
  
  <p>In any case, based upon this conservative linear fit, the authors predict that the number of neurons in a ANN will equal that of the human brain in 2056.</p>
  
  <p>The referenced nerual networks are:</p>
  
  <p><a href="https://i.sstatic.net/f8Y6O.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/f8Y6O.png" alt="Neural networks referred to in image above"></a></p>
  
  <p>What is interesting to note that when <a href="https://en.wikipedia.org/wiki/The_Singularity_Is_Near" rel="nofollow noreferrer">The Singularity is Near</a> was written in 2006, Ray Kurzweil said that the refractory period of a biological neuron was already 1,000,000 times slower than that of an artificial one.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-25T19:12:07.523Z
Answers.Text: @TomHale - regarding - "Ray Kurzweil said that the refractory period of a biological neuron was already 1,000,000 times slower than that of an artificial one. " - does that mean the artificial neurons will be able to do more computations?
Answers.UserDisplayName: committedandroider
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-26T08:06:58.950Z
Answers.Text: @committedandroider if the complexity of an artificial neuron could be increased to that of a biological one, while keeping the same speed multiplier, then I don't see why not. Also, artificial neurons could be electrical whereas there is a slower chemical component to brain neurons, AFAIK.
Answers.UserDisplayName: Tom Hale
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-14T12:47:01.827Z
Answers.Text: @Keno is referring to [this answer](http://ai.stackexchange.com/a/2334/2424)
Answers.UserDisplayName: Tom Hale
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-22T19:46:06.937Z
Answers.Text: Revised my table numbers down, so year 2035 is my current estimate.
Answers.UserDisplayName: Keno
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-14T07:29:24.257Z
Answers.LastActivityDate: 2016-11-14T07:35:01.833Z
Answers.LastEditDate: 2016-11-14T07:35:01.833Z
Answers.UserDisplayName: Tom Hale
Answers.Body: <p>While interesting, this is all rendered somewhat moot if you think about what will happen once we understand how the brain works. After all, once we understood flight, we didn't start making birds. The same goes for AI. Here are just a few ways in which human brains and digital brains can't be compared. </p>
  
  <p>The digital brain won't have to worry about food and drink. They will also be more reliable (or less redundant) as electronics is way more reliable than neurons (a guess). Digital brains will also be able to share learning and information. Once one Model 3X digital brain has learned something, the others need merely to have the bits uploaded. Sure, it will be more complicated than that but, remember, we will know how it all works so merging the experiences of one digital being with another should be doable. If we want our digital brain to have symbolic algebra ability, we will have to teach it some things but we can also hard-wire it to Mathematica or the like. </p>
  
  <p>In short, it will be like apples and oranges.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-09T10:00:19.790Z
Answers.Text: _electronics is way more reliable than neurons_ - I'd love to see a complex electronic system that can function nonstop for nearly a century with absolutely zero downtime (general anesthesia notwithstanding).
Answers.UserDisplayName: forest
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-25T19:14:17.977Z
Answers.Text: I was thinking about this too - "The digital brain won't have to worry about food and drink." . If we don't have to model every human behavior, I'am interested in how neurons we would need to model the human brain
Answers.UserDisplayName: committedandroider
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-29T00:44:55.590Z
Answers.Text: Nice answer.  Welcome to AI!  There is a lot of interest in translating/migration human minds to computing mediums in a hypothetical post-singularity context.  Wondered if you had any thoughts on the idea of maintaining a software kernel based on human brain for whatever reason, even if that model becomes technically obsolete in terms of raw intelligence.
Answers.UserDisplayName: DukeZhou
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-29T00:16:27.760Z
Answers.LastActivityDate: 2017-08-29T00:16:27.760Z
Answers.UserDisplayName: Paul Topping
Answers.Body: <p>Soon enough but that doesn't mean anything at all. In machine learning the word neuron represents a calculation whereas in brain the word neuron represent a specific type of cell which is a biochemical system.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-14T06:50:51.683Z
Answers.Text: Yes, plus multiple calculations are carried out within biological neurons through the interactions of a dizzying array of neurotransmitters etc. It might take multiple hardware or software neurons to get the same effect in an artificial neural net. Furthermore, we don't know how many of these neurochemical exchanges of information are taking place - new ones are being discovered all the time - and have a poor understanding of what their functions are. So we can't even give an accurate number of how many processing units we need, let alone estimate when we'll have them.
Answers.UserDisplayName: SQLServerSteve
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-14T07:39:15.627Z
Answers.Text: A biochemical system that performs performs a complex calculation with a  chemical return value...
Answers.UserDisplayName: Tom Hale
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-14T11:46:18.760Z
Answers.Text: @TomHale What is your reasoning that made you reach the conclusion that the biochemical system is performing a calculation? Also, given any system how would you decide if it is performing some calculation or not?
Answers.UserDisplayName: Ankur
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-14T12:45:43.663Z
Answers.Text: It's performing a transformation of inputs based on its current biological state and the "rules" of the biological processes, and produces outputs
Answers.UserDisplayName: Tom Hale
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-14T14:38:06.823Z
Answers.Text: @TomHale: Ok, now consider this scenario - I have a bucket with bunch of holes at the bottom, now if I start pouring water in the bucket, the water will start to come out of the holes at the bottom - There is input and there is output. Would you consider the bucket doing computation?
Answers.UserDisplayName: Ankur
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-14T15:08:10.153Z
Answers.Text: @Ankur I should have said function. You describe a rate limiter function. I agree that your wording "computation" is going too far.
Answers.UserDisplayName: Tom Hale
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-15T03:44:33.327Z
Answers.Text: @TomHale: That's the problem I have with associating the word computation to any causal process. If there is no experiment that I can use to test whether a causal process is computation or not than it doesn't provide no useful insight to say that neurons are doing computation.
Answers.UserDisplayName: Ankur
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-14T05:57:36.463Z
Answers.LastActivityDate: 2016-11-14T05:57:36.463Z
Answers.UserDisplayName: Ankur
Answers.Body: <p>The human brain contains billions of neurons, which means we won't be making one tomorrow. However, technology tends to advance in an exponential manner, and that may soon be a real possibility. Also, the idea of making an artificial human brain would not only take more neurons than a current average computer could process, or we could make outside of computers, but we also need an understanding of the human brain. There is only one animal with neurons that we have completed a full connectome of and that is the Caenorhabditis elegans (roundworm) and it has less than 500 neurons. It may be a while before we actually make a human brain, but within 30 years is a reasonable estimation with the rate that technology improves now.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-25T19:07:11.213Z
Answers.Text: Is there a list that maps out how many neurons each animal has. I'am thinking we could work our own way up from the roundworm to the fruit fly then to a rat...
Answers.UserDisplayName: committedandroider
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-28T04:17:15.653Z
Answers.Text: Welcome to AI.SE :)
  
  Please note that you didn't actually answer the question... please re-read the first 10 words.
Answers.UserDisplayName: Tom Hale
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-29T01:44:17.487Z
Answers.Text: @TomHale oops thanks I didn't realize. I'll fix that when I get a chance. Thanks for pointing it out. :)
Answers.UserDisplayName: I. Golsby
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-27T16:38:38.143Z
Answers.LastActivityDate: 2016-11-27T16:38:38.143Z
Answers.UserDisplayName: I. Golsby
Answers.Body: <p>2035, 2056? Those predictions are hilarious :)</p>
  <p>2019 - 1,6Billion parameter model (GPT-2)</p>
  <p>2020 -175Billion parameter model (GPT-3) more than 100x jump in a year</p>
  <p>2021(April) - &quot;Microsoft's ZeRO-Infinity can now run a model with over a trillion parameters on a single NVIDIA DGX-2 node and over 30 trillion parameters on 32 nodes (512 GPUs). With a hundred DGX-2 nodes in a cluster, Microsoft projects ZeRO-Infinity can train models with over a hundred trillion parameters&quot;</p>
  <p><a href="https://www.microsoft.com/en-us/research/blog/zero-infinity-and-deepspeed-unlocking-unprecedented-model-scale-for-deep-learning-training/" rel="nofollow noreferrer">https://www.microsoft.com/en-us/research/blog/zero-infinity-and-deepspeed-unlocking-unprecedented-model-scale-for-deep-learning-training/</a></p>
  <p>So in 2021 we now have tech to train 30Trillion- 100 Trillion parameters/neurons Model</p>
  <p>100 trillion = Human Brain</p>
  <p>With this tech, OpenAI together with Microsoft, other company or gov will train 100Trillion model in 2021 or 2022 at the latest</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-23T14:33:13.973Z
Answers.Text: Can you provide a reference that supports this claim "100 trillion = Human Brain"?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-23T13:08:35.453Z
Answers.LastActivityDate: 2021-04-23T13:08:35.453Z
Answers.UserDisplayName: Patrick
Body: <p>Based on fitting to historical data and extrapolation, when is it expected that the number of neurons in AI systems will equal those of the human brain?</p>
  
  <p>I'm interested in a possible direct replication of the human brain, which will need equal numbers of neurons.</p>
  
  <p>Of course, this assumes neurons which are equally capable as their biological counterparts, which development may happen at a faster or slower rate than the quantitative increase.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-11-14T14:27:58.670Z
Comments.Text: Can you add information about why you think this is importan-- toward what end does the answer serve? I do see some useful information in answers below, but to date I think the best response is @Ankur 's, which points out the answer is without utility unless we define what it means to have a certain number of neurons.
Comments.UserDisplayName: dynrepsys
Tags: <neurons><prediction>
Title: When will the number of neurons in AI systems equal the human brain?
UserDisplayName: Tom Hale
ThreadText: # When will the number of neurons in AI systems equal the human brain?
  
  (Asked by: username_0 on 2016-11-14)
  
  Based on fitting to historical data and extrapolation, when is it expected that the number of neurons in AI systems will equal those of the human brain?
  
  I'm interested in a possible direct replication of the human brain, which will need equal numbers of neurons.
  
  Of course, this assumes neurons which are equally capable as their biological counterparts, which development may happen at a faster or slower rate than the quantitative increase.
  
  **Tags:** neurons, prediction
  > **Comments:**
  > **username_1**: Can you add information about why you think this is importan-- toward what end does the answer serve? I do see some useful information in answers below, but to date I think the best response is @Ankur 's, which points out the answer is without utility unless we define what it means to have a certain number of neurons.  
  
  ---
  
  ## 6 answers
  
  ### Answer by username_2 on 2016-11-14. Score: 17
  
  Some back of the envelope calculations :
  
  number of neurons in AI systems 
  
  The number of neurons in AI systems is a little tricky to calculate, Neural Networks and Deep Learning are 2 current AI systems as you call them, specifics are hard to come by (If someone has them please share), but data on parameters do exist, parameters are more analogous to synapses (connections) than neurons (the nodes in between connections) somewhere in the range of 100-160 billion is the current upper number for specialized networks.
  Deriving the number of neurons in AI systems from this number is a stretch since these AIs emulate certain types of connections and sub assemblies of neurons, but let's continue...
  
  equal those of the human brain?
  
  So now let's look at the brain, and again this are all contested numbers. Number of neurons ~ 86 Billion, Number of Synapses ~ 150 Trillion, another generalization: average number of synapses per neuron ~ 1,744.
  So now we have something to compare, and I can't stress this enough, these are all wonky numbers, so let's make our life a little easier and divide :
  Number of Synapses (Brain ) : 150 trillion /  Number of parameters AIs : 150 billion = 1,000 or in other words current AIs would have to scale by a factor of one thousand their connections to be on par with the brain...
  Number of Neurons (Brain ) : 86 Billion / Number of Neurons AIs ( 150 billion / 1,744 )  = 86 Million equivalent AI Neurons
  Which makes sense, mathematically at least : you can multiply the factor ( 1000 ) times the current number of equivalent AI Neurons ( 86 million) to get the number of neurons in the human brain (86 Billion)
  
  When ?
  
  Well,let's use  moore's law ( number of transistors processing power doubles about every 2 years ) as a rough measure of technological progress: 
       #AI NEURONS            YEAR
       86,000,000             2016
       172,000,000            2018
       344,000,000            2020
       688,000,000            2022
       1,376,000,000          2024
       2,752,000,000          2026
       5,504,000,000          2028
       11,008,000,000         2030
       22,016,000,000         2032
       44,032,000,000         2034
       88,064,000,000         2036
  
  
       # NEURONS HUMAN BRAIN 
       86,000,000,000
  
  So, if all this made sense to you, somewhere around the year 2035.
  
  > **Comments:**
  > **username_3**: https://arxiv.org/abs/1701.06538 , this might incline you revise your estimate. ;-) They want to scale up to a trillion parameters and I wouldn't be surprised if they did it this year, which would be a jump to 2022 on your timeline.  
  > **username_4**: is the number of parameters in this analogy equivalent to the number of synapses?  
  > **username_2**: @committedandroider: roughly but yes, parameters here are weights in between neurons/nodes in ANN, in natural NN the equivalent is synaptic weights, which map to synapses.  
  
  ---
  
  ### Answer by username_8 on 2016-11-14. Score: 7
  
  Soon enough but that doesn't mean anything at all. In machine learning the word neuron represents a calculation whereas in brain the word neuron represent a specific type of cell which is a biochemical system.
  
  > **Comments:**
  > **username_9**: Yes, plus multiple calculations are carried out within biological neurons through the interactions of a dizzying array of neurotransmitters etc. It might take multiple hardware or software neurons to get the same effect in an artificial neural net. Furthermore, we don't know how many of these neurochemical exchanges of information are taking place - new ones are being discovered all the time - and have a poor understanding of what their functions are. So we can't even give an accurate number of how many processing units we need, let alone estimate when we'll have them.  
  > **username_0**: A biochemical system that performs performs a complex calculation with a  chemical return value...  
  > **username_8**: @TomHale What is your reasoning that made you reach the conclusion that the biochemical system is performing a calculation? Also, given any system how would you decide if it is performing some calculation or not?  
  > **username_0**: It's performing a transformation of inputs based on its current biological state and the "rules" of the biological processes, and produces outputs  
  > **username_8**: @TomHale: Ok, now consider this scenario - I have a bucket with bunch of holes at the bottom, now if I start pouring water in the bucket, the water will start to come out of the holes at the bottom - There is input and there is output. Would you consider the bucket doing computation?  
  > **username_0**: @Ankur I should have said function. You describe a rate limiter function. I agree that your wording "computation" is going too far.  
  > **username_8**: @TomHale: That's the problem I have with associating the word computation to any causal process. If there is no experiment that I can use to test whether a causal process is computation or not than it doesn't provide no useful insight to say that neurons are doing computation.  
  
  ---
  
  ### Answer by username_0 on 2016-11-14. Score: 3
  
  The answers so far haven't answered the question numerically, so here is my attempt to steer them in the direction I was seeking:
  
  The freely available [Deep Learning Book](http://www.deeplearningbook.org) has the following figure on page 27:
  
  [![Size of neural nets over time](https://i.sstatic.net/iz2C4.png)](https://i.sstatic.net/iz2C4.png)
  
  I question the blue fit line, as it seems that data points may be better described by a parabolic or exponential function.
  
  In any case, based upon this conservative linear fit, the authors predict that the number of neurons in a ANN will equal that of the human brain in 2056.
  
  The referenced nerual networks are:
  
  [![Neural networks referred to in image above](https://i.sstatic.net/f8Y6O.png)](https://i.sstatic.net/f8Y6O.png)
  
  What is interesting to note that when [The Singularity is Near](https://en.wikipedia.org/wiki/The_Singularity_Is_Near) was written in 2006, Ray Kurzweil said that the refractory period of a biological neuron was already 1,000,000 times slower than that of an artificial one.
  
  > **Comments:**
  > **username_0**: @Keno is referring to [this answer](http://ai.stackexchange.com/a/2334/2424)  
  > **username_2**: Revised my table numbers down, so year 2035 is my current estimate.  
  > **username_4**: @TomHale - regarding - "Ray Kurzweil said that the refractory period of a biological neuron was already 1,000,000 times slower than that of an artificial one. " - does that mean the artificial neurons will be able to do more computations?  
  > **username_0**: @committedandroider if the complexity of an artificial neuron could be increased to that of a biological one, while keeping the same speed multiplier, then I don't see why not. Also, artificial neurons could be electrical whereas there is a slower chemical component to brain neurons, AFAIK.  
  
  ---
  
  ### Answer by username_5 on 2017-08-29. Score: 3
  
  While interesting, this is all rendered somewhat moot if you think about what will happen once we understand how the brain works. After all, once we understood flight, we didn't start making birds. The same goes for AI. Here are just a few ways in which human brains and digital brains can't be compared.
  
  The digital brain won't have to worry about food and drink. They will also be more reliable (or less redundant) as electronics is way more reliable than neurons (a guess). Digital brains will also be able to share learning and information. Once one Model 3X digital brain has learned something, the others need merely to have the bits uploaded. Sure, it will be more complicated than that but, remember, we will know how it all works so merging the experiences of one digital being with another should be doable. If we want our digital brain to have symbolic algebra ability, we will have to teach it some things but we can also hard-wire it to Mathematica or the like.
  
  In short, it will be like apples and oranges.
  
  > **Comments:**
  > **username_7**: Nice answer.  Welcome to AI!  There is a lot of interest in translating/migration human minds to computing mediums in a hypothetical post-singularity context.  Wondered if you had any thoughts on the idea of maintaining a software kernel based on human brain for whatever reason, even if that model becomes technically obsolete in terms of raw intelligence.  
  > **username_6**: _electronics is way more reliable than neurons_ - I'd love to see a complex electronic system that can function nonstop for nearly a century with absolutely zero downtime (general anesthesia notwithstanding).  
  > **username_4**: I was thinking about this too - "The digital brain won't have to worry about food and drink." . If we don't have to model every human behavior, I'am interested in how neurons we would need to model the human brain  
  
  ---
  
  ### Answer by username_10 on 2016-11-27. Score: 1
  
  The human brain contains billions of neurons, which means we won't be making one tomorrow. However, technology tends to advance in an exponential manner, and that may soon be a real possibility. Also, the idea of making an artificial human brain would not only take more neurons than a current average computer could process, or we could make outside of computers, but we also need an understanding of the human brain. There is only one animal with neurons that we have completed a full connectome of and that is the Caenorhabditis elegans (roundworm) and it has less than 500 neurons. It may be a while before we actually make a human brain, but within 30 years is a reasonable estimation with the rate that technology improves now.
  
  > **Comments:**
  > **username_0**: Welcome to AI.SE :)  Please note that you didn't actually answer the question... please re-read the first 10 words.  
  > **username_10**: @TomHale oops thanks I didn't realize. I'll fix that when I get a chance. Thanks for pointing it out. :)  
  > **username_4**: Is there a list that maps out how many neurons each animal has. I'am thinking we could work our own way up from the roundworm to the fruit fly then to a rat...  
  
  ---
  
  ### Answer by username_11 on 2021-04-23. Score: 0
  
  2035, 2056? Those predictions are hilarious :)
  
  2019 - 1,6Billion parameter model (GPT-2)
  
  2020 -175Billion parameter model (GPT-3) more than 100x jump in a year
  
  2021(April) - "Microsoft's ZeRO-Infinity can now run a model with over a trillion parameters on a single NVIDIA DGX-2 node and over 30 trillion parameters on 32 nodes (512 GPUs). With a hundred DGX-2 nodes in a cluster, Microsoft projects ZeRO-Infinity can train models with over a hundred trillion parameters"
  
  <https://www.microsoft.com/en-us/research/blog/zero-infinity-and-deepspeed-unlocking-unprecedented-model-scale-for-deep-learning-training/>
  
  So in 2021 we now have tech to train 30Trillion- 100 Trillion parameters/neurons Model
  
  100 trillion = Human Brain
  
  With this tech, OpenAI together with Microsoft, other company or gov will train 100Trillion model in 2021 or 2022 at the latest
  
  > **Comments:**
  > **username_12**: Can you provide a reference that supports this claim "100 trillion = Human Brain"?  
  
  ---
  
  