Body: <p>It just struck me that the human brain has <a href="https://en.wikipedia.org/wiki/Human_brain#Microanatomy" rel="nofollow noreferrer">about as many neurons</a> as an <a href="https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor-core-gpu-datasheet?ncid=no-ncid" rel="nofollow noreferrer">Nvidia M100 chip has transistors</a>, around 8E10. Obviously, a transistor is not equivalent to a neuron; but Elon Musk's xAI is <a href="https://x.com/elonmusk/status/1810727394631950752" rel="nofollow noreferrer">planning to build a cluster with 100,000 such processors</a>.</p>
  <p>I'm not familiar with the hardware topology of such clusters; but would it be fair to say that this cluster has a &quot;complexity&quot; (however vaguely defined) in the order of magnitude of the human brain?</p>
  <p>There is a caveat here in that comparing computational hardware with biological wetware, like in the above question, is comparing apples to pears: The brain's physical basis <em>is</em> a neural net in the truest sense of the word while the computational hardware of xAI's cluster will <em>run</em> an emulation of one, even though it is dedicated hardware supporting this specific purpose.</p>
  <p>I hope it's OK to ask this related question in the same post: Is the complexity of the <em>model</em> such a cluster might run/train comparable to the brain's neural network?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-10T20:13:51.513Z
Comments.Text: you should also consider the connectivity, and then realize there is no such thing in computers, if you don't specify a model that exploits the computation
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-10T20:57:36.940Z
Comments.Text: @Alberto Hm... there clearly is connectivity in computers (i.e., in their hardware).
Comments.UserDisplayName: Peter - Reinstate Monica
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-10T21:41:04.620Z
Comments.Text: what do you mean? you don't need electricity to make computers, you can make turing machines out of water and tubes, but you would never say that the world water infrastructure has "the complexity of a brain"
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-11T07:15:31.533Z
Comments.Text: @Alberto The national water infrastructure of an industrialized country ([here](https://media.diercke.net/omeda/501/100260_021_6.jpg) is an image of the water backbone network for a German state) may be of comparable complexity as the connectome of a nematode, I think. Electricity networks, being meshed more, will have a greater complexity. The *topology* likely differs dramatically, sure (most distribution networks will be more hierarchical than a connectome, I suppose), but complex they are.
Comments.UserDisplayName: Peter - Reinstate Monica
Tags: <artificial-neuron><computational-complexity><neurons><brain>
Title: A 100,000 M100 cluster: Complexity of the human brain?
UserDisplayName: Peter - Reinstate Monica
ThreadText: # A 100,000 M100 cluster: Complexity of the human brain?
  
  (Asked by: username_0 on 2024-07-10)
  
  It just struck me that the human brain has [about as many neurons](https://en.wikipedia.org/wiki/Human_brain#Microanatomy) as an [Nvidia M100 chip has transistors](https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor-core-gpu-datasheet?ncid=no-ncid), around 8E10. Obviously, a transistor is not equivalent to a neuron; but Elon Musk's xAI is [planning to build a cluster with 100,000 such processors](https://x.com/elonmusk/status/1810727394631950752).
  
  I'm not familiar with the hardware topology of such clusters; but would it be fair to say that this cluster has a "complexity" (however vaguely defined) in the order of magnitude of the human brain?
  
  There is a caveat here in that comparing computational hardware with biological wetware, like in the above question, is comparing apples to pears: The brain's physical basis *is* a neural net in the truest sense of the word while the computational hardware of xAI's cluster will *run* an emulation of one, even though it is dedicated hardware supporting this specific purpose.
  
  I hope it's OK to ask this related question in the same post: Is the complexity of the *model* such a cluster might run/train comparable to the brain's neural network?
  
  **Tags:** artificial-neuron, computational-complexity, neurons, brain
  
  > **Comments:**
  > **username_1**: you should also consider the connectivity, and then realize there is no such thing in computers, if you don't specify a model that exploits the computation  
  > **username_0**: @username_1 Hm... there clearly is connectivity in computers (i.e., in their hardware).  
  > **username_1**: what do you mean? you don't need electricity to make computers, you can make turing machines out of water and tubes, but you would never say that the world water infrastructure has "the complexity of a brain"  
  > **username_0**: @username_1 The national water infrastructure of an industrialized country ([here](https://media.diercke.net/omeda/501/100260_021_6.jpg) is an image of the water backbone network for a German state) may be of comparable complexity as the connectome of a nematode, I think. Electricity networks, being meshed more, will have a greater complexity. The *topology* likely differs dramatically, sure (most distribution networks will be more hierarchical than a connectome, I suppose), but complex they are.  
  
  ---
  
  ## 0 answers
  
  