Superintelligence cannot be contained: Lessons from Computability Theory
I haven't read this properly yet, but a skim leaves me skeptical. For example:> "Another lesson from computability theory is the following:
we may not even know when superintelligent machines have
arrived, as deciding whether a machine exhibits intelligence
is in the same realm of problems as the containment problem. This is a consequence of Rice’s theorem [24], which
states that, any non-trivial property (e.g. “harm humans” or
“display superintelligence”) of a Turing machine is undecidable"One man's modus ponens is another man's modus tollens. If their theory says that superintelligence is not recognisable, then they're perhaps not using a good definition of superintelligence, because obviously we will be able to recognise it.