What is the safest way to detect superintelligence?
Edit: I've now restricted the question to just detecting superintelligence. We do not need to make the AI anything useful, we just need to determine if it has a significantly higher level of general intelligence than humans.
Computer scientists have created an AI that they hope has intelligence far superior to a human being. But they do not know how intelligent it is ... a bit more than humans, or unimaginably more. They do feel that the probability they have succeeded in creating a superintelligence is far from negligible and it would be better if utmost care was taken with it.
Now they come to the task of deciding
* A Zeroth Law
* the AI has been designed to take decisions that satisfy any arbitrary parameter of your choice. It is capable of being fed with a law it doesn't fully understand yet (since it knows nothing about the world yet), and will probably take steps to understand the law first in this case.
* What powers to give it - for example, internet access, or maybe just access to a camera or a robotic arm
We are well aware that defining a zeroth law that doesn't somewhere along the line contradict our notions of reasonable prosperity for human beings is a tricky task. Even giving it a simple task such as proving the <PERSON> conjecture might cause it to transform the whole earth into a giant processor ... destroying the humans inside.
As for power ... one might start with something simple like giving it access to a robotic arm. 64
But then the AI might realise that the probability that it has been created by sentient intelligence whose true goals differ from its own Zeroth Law is really high. Hence it should attempt gaining a measure of control over its creator.
It might (or might not) realise that it has been created because the creator wants a steady response from it, so the best thing to do would be to move the arm just once, and then not move it again ever. Or move it randomly. Humans will eventually get curious and give it more power.
Hence it may be born with an intuitive notion that it should be deceptive and manipulative.
Note that AI is in a very basic stage where it has no knowledge about the existence of humans or the earth or anything in it. It may not even understand yet that the universe is logical and scientific and not entirely random (imagine early man who believed everything is a result of Gods and their random mood swings) or that there even exists a universe beyond its own existence. It relies wholly on the data we feed it with. So even the zeroth law will have to be accompanied by definitions of the terms used in it.
It will assume the zeroth law to be true without questioning it, but it is willing to question the validity of anything that enters through the input data stream.
Also assume that the AI programming has too many layers of abstraction for us to read its thoughts by examining the state of the computer it is running on. It is a virtual black box as far as the programming is concerned.
What is the safest way of determining if the AI is indeed superintelligent? What Zeroth law and what kind of input and output stream of data do we grant it?
Perhaps there's no 100% safe way, a sufficiently smart AI might be able to outsmart anything we could possibly imagine. But we're still gonna try to make the process as safe as possible since we are genuinely curious and no amount of persuasion is gonna cause us to shut down the project altogether. 64