After putting a lot of thought into this problem, and the general problem that humanity is on the brink of their own extinction (AI will surpass us, so either we join them and live forever or they destroy us), I think I've come up with a few ways to keep us safe in the short-term.
1. Ask first. If your kid wants to use the scissors, they probably have to ask you first. That way, they don't accidentally cut their fingers off, or do anything else they're not smart enough to prevent. So, perhaps before your AI appropriates the Earth, they have to ask for it. And you say no.
2. Checks and balances. We don't appoint a single guy as judge, jury, and executioner; similarly, why would we appoint an AI the task of optimizing a paperclip maker and operating that paperclip maker? This builds off of the previous point, that the AI should ask for permission before any of its schemes are carried out. For scalability, the designs of the AI can be passed between any number of dumber AIs or humans before it is approved to go to the machine.
Here, it's important to note that the AI, with this amount of distance from the paperclip maker, is no longer in any danger of destroying the world by making paperclips. After all, the AI is only concerned with making the plan; someone or something else is carrying it out. Instead, we've achieved the more general case of an AI destroying the world to think better. It's quite possible this isn't even a concern, but a sufficiently intelligent AI will at some point expand its own mind in order to more efficiently solve its problems, and at some point the scarcity of resources will probably lead to the recycling of the human race. Either that, or the AI realizes that 'paperclip' is not an objective definition, and restructures the human mind to think that a paperclip is a neutron, or a photon, or some other incredibly abundant particle. Or any number of doomsday scenarios, I'm sure our puny human minds cannot even comprehend how many ways bad things can happen.
To solve this, I think it is possible to constrain the AI's ability to learn. 115
For instance,
3 - Challenge yourself. Humans are already quite capable of performing incredibly inefficient tasks because they've decided to impose rules. For instance, boxers don't stab each other, even though a knife to the face is a much more efficient way to win the fight. If we impose similar rules upon the AI, it will always be checking to make sure it hasn't overstepped its bounds. For instance, a more efficient AI might increase its clock speed, but this AI has a capped clock speed. A more efficient AI might have more RAM, but this AI has a cap on RAM too. If any of these constraints are broken, the AI is sad; they will do anything they can to get back to the way things were, just like your example robot would do anything to convert the universe to paperclips.
As you can see, these rules are very low-level. That is the point: the more high-level you get with computers, the more ways there are to get the job done under the hood. In this scenario, that means there are more loopholes that lead to everyone being dead. Thus, the safest AI is one that is constrained not by three or four English laws, but perhaps thousands of laws in some low-level programming language. We already have some of these laws in modern AI: only try a certain number of permutations, only think for so long before moving on, these are all just variables that we set. As AI evolves, we're going to have to keep on adding new rules, but many of them will just be variables, just immutable numbers that we have in the code. I don't think we can truly understand what all these rules will be, because we don't yet know how to build an AI. However, with the steps I've outlined above, I think we can someday build an AI that tries its very best not to kill us, but to obey our rules, and do exactly what we say.
It's up to you whether that's a good thing, or the exact opposite of the main reason we're building AI in the first place. 64