Should AI Be Open?
So, here&#x27;s a random thought on this whole subject of &quot;AI risk&quot;.<p>Bostrom, Yudkowsky, etc. posit that an &quot;artificial super-intelligence&quot; will be many times smarter than humans, and will represent a threat somewhat analogous to an atomic weapon. BUT...  consider that the phrase &quot;many times smarter than humans&quot; may not even mean anything.   Of course we don&#x27;t know one way or the other, but it seems to me that it&#x27;s possible that we&#x27;re already roughly as intelligent as it&#x27;s possible to be.  Or close enough that being &quot;smarter than human&quot; does <i>not</i> represent anything analogous to an atomic bomb.<p>So this might be an interesting topic for research, or at least for the philosophers: &quot;What&#x27;s the limit of how &#x27;smart&#x27; it&#x27;s possible to be&quot;?  It may be that there&#x27;s no possible way to determine that (you don&#x27;t know what you don&#x27;t know and all that) but if there is, it might be enlightening.