text: Later in the evening, Kathy Schulz called. She wanted to know how the investigation was going and what the board had decided to do. I gathered she wasn't too happy about being made to stay at home.
  
  "Andy, before I go, I need to tell you something. I… I hope you're right that it wasn't Raphael on Sunday night. But if it was him, then we have a big problem on our hands. And I'm not talking about having to start over again." She hesitated. "We have to start thinking about what else can he do out there. We may have to make news of this public. There's no telling—"
  
  "Kathy, not you too!" I said, letting out a groan. "Unless you can tell me how the containment could have failed, don't bother painting doomsday scenarios. I've had more than my fill today."
  
  "Andy, there could be a way to circumvent the First Directive."
  
  "I'm listening."
  
  I felt her hesitating at the other end.
  
  "It concerns Moore's Paradox."
  
  "That old bugbear again? We've hashed it out, Kathy—at length, when we were designing the directives. It doesn't apply."
  
  "We may have been wrong."
  
  Moore's Paradox was a particular problem in logic that involved Moorean sentences; sentences like: It is raining, but I don't believe it is raining. While it may sound odd, there is actually nothing wrong with the sentence. It is logically consistent, and it could be true, since it's perfectly possible for one to believe something that's false (to believe that it's not raining when in fact it is). But when you say the sentence out loud, it seems absurd. The paradox is how to explain the seeming absurdity of a logically consistent and possibly true sentence.
  
  A common explanation is to observe that the first half of the sentence isn't just a statement of fact, but also a belief. When you say it is raining, you are expressing a belief that it is raining. Therefore, it would be contradictory to say both "I believe it is raining" and "I don't believe it is raining" in the same sentence.
  
  Some of the designers thought the paradox was relevant because of the way we modeled Raphael's directives as beliefs. They felt there was a chance a contrary belief could arise at some point in Raphael's mind, and when juxtaposed with the original belief, lead to a paradox. In case of the First Directive, a Moorean sentence would look something like: I cannot change my directives but I believe I can. Such a paradoxical situation could lead to unpredictable behavior, where sometimes the first belief overruled the second, and sometimes the second the first.
  
  But the very structure of his brain prevented such paradoxes. The beliefs didn't exist in isolation: they formed an interconnected web. Newly arrived beliefs would survive in this ecosystem of beliefs only if they cohered, or "got along well", with the web as a whole. To survive, they had to be justified by pre-existing beliefs. In general, the older a particular belief was, the more interconnected and "entrenched" in the web it would be, and the harder it would be to replace it, with the very first beliefs—such as the ones in the commandment chip—forming a virtual scaffolding that held the entire structure together. A bit like a deeply religious person brought up on the idea of a personal god finding it hard to accept the idea of an indifferent universe. The Moorean sentence, I cannot change the directives but I believe I can, could not create a paradoxical situation simply because the first half of the sentence was a deeply entrenched belief. If a contradictory belief ever arose in his mind, it would fail to cohere with the belief net and be quickly eliminated.
  
  I said to Kathy, "Moore's Paradox doesn't apply. A belief counter to the First Directive cannot take root in the belief net because strong, pre-existing beliefs always overrule the weaker, contradictory new belief. You know we have tested this many times."
  
  "We never tested for subnets," she replied.
  
  "Subnets…"
  
  "A contradictory belief could survive deep inside a subnet."
  
  She was talking about a separate network of ideas and beliefs, only loosely connected to the larger web. If the subnet was extensive and deep enough, it was theoretically possible for strange, otherwise contradictory ideas to exist at the center of this smaller web. These extreme beliefs only had to cohere with their immediate neighbors, which would be less extreme, and so on until you reached the outer edges of the subnet where the connection points were. These would be more or less in tune with the philosophy of the larger net.
  
  Like a tumor that goes undetected by disguising itself.
  
  "You are talking about a whole new system of beliefs. Maybe even a split personality."
  
  "Yes."
  
  "We would have known. Something like that involves a pretty big neuronal reorganization. It would have shown in the scans."
  
  "We haven't been doing scans for more than a month, Andy."
  
  "A month's not enough time for such a drastic change. He'd have to—"
  
  "Not enough for a human," she interrupted. "You've seen how fast he learns. This whole month we left him alone he could have been quietly reinforcing those ideas over and over again."
  
  "I don't know, Kathy…"
  
  "We have to find out what he's been reading, who he's been talking to in the last few weeks. We'll have to go through the tapes and find out if—"
  
  The rest of her words were lost to me because something in my mind was clamoring for attention. She was wrong about the subnet theory of course, but she'd said something that had set off a chain of thought. A possible way out of the mess. I'd have to think about it some more. It would mean changing—
  
  "So you okay with it? Andy…?"
  
  "Sorry, what was that last part?"
  
  "Are you okay with me telling Valery about it? I'll have to go to the lab. Check out the tapes—see what he was exposed to last month."
  
  "Go ahead," I said distractedly. My mind didn't register what followed next except the tone at the end telling me that she'd hung up.