text: We sat in silence for a few seconds, the image of Raphael holding up the can frozen on the screen.
  
  "Still having trouble believing, Andy?" Troy quipped.
  
  "I told you, I'm going to hold off theorizing until I've seen everything. Where's the recording of the actual robbery?"
  
  Martinez replied, "We don't have it."
  
  "What do you mean you don't have it?"
  
  "Exactly what I said, Dr. Ahuja. There is no recording of the core being taken."
  
  I frowned at the screen. "I realize that some of the cameras inside were blinded, but surely the ones outside got a look the thieves?"
  
  "We checked all the recordings right until they end on Monday morning, when Dan took them out. Not a single camera shows anyone entering or leaving the premises. Except the guards changing shifts, that is. Building maintenance finishes by Saturday afternoon, so there were no cleaning crews either."
  
  "It can't be… Are you sure you checked everything?" I said.
  
  "Twice," Dan said, his eyes travelling to Martinez. "Main doors, fire exits, atrium, lift lobbies, the rooftop…everywhere. If the tapes are to be believed, there was no robbery."
  
  "This doesn't make sense. What about ventilation ducts? Windows? There must be some way they got in."
  
  "No and no. The only way into the ventilation ducts is through the basement or the rooftop. The cameras there show nothing out of the ordinary. After we took over, I installed motion sensors and zappers in the ducts… for rats. They would have registered if something went past them. As for the windows, they can't be opened from the outside. And there are no signs of forced entry."
  
  "Then the core is still in the lab!" I cried. "It must be! That's the only logical explanation."
  
  Dan shook his head. "I assure you it's not. Sheng and I were the first ones in and I sealed off the floor as soon as I realized what had happened. We searched everywhere. It's really gone."
  
  "I suggest you look harder. There m—"
  
  Martinez interrupted me. "We were hoping you might have an explanation."
  
  "How's that?"
  
  "We saw Raphael opening his chest cavity. We assumed it was so that the thieves could quickly retrieve the core and leave. There is another explanation. He could have taken out the core himself and thrown it out of a window to someone waiting below. Dan doesn't agree with me, though."
  
  "The windows can't be opened without triggering the alarms," Dan said. "And I can confirm that no alarms went off."
  
  "There's another reason why it's not possible," I said. "Imagine surgically removing your brain, holding it in your hands and walking across the room. Can't be done. It's the same with Raphael. He would lose all control over his body the moment he unplugged the connectors to the core's ports. Not to mention running the risk of suffering irreversible brain damage if the core wasn't shut down. The core was shut down last Friday. The logs don't lie."
  
  "Logs can be manipulated," Troy interjected.
  
  "Dan, do you want to answer that or shall I?" I said testily.
  
  Dan cleared his throat. "Um, what Andy is trying to say is that the logs are read-only."
  
  I explained further. "After Halicom took over Mirall, Dan applied a group policy to all company devices, computers, and phones, making all important logging read-only. Only the system admin has access to change or delete any logs." I did not see it necessary to remind them who the system admin was. "We are in a bind," I said. "You say the core is not in the lab. Yet you are equally sure no one entered the building."
  
  "Actually, that's not entirely correct," Dan said. "There was that incident with the guard…"
  
  He was rewarded with an angry look from Martinez. "Er, did you not want me to bring it up?"
  
  "Bring what up?" I said.
  
  Like a puppy that's just crapped all over the sofa, Dan looked to her for direction.
  
  "We may have caught one of the robbers on our cameras. We are not sure," Martinez said, still shooting daggers at Dan. "It's from the atrium below, recorded a few minutes after the incident in the server room. I admit the timing is too close to be a coincidence, yet—"
  
  "Just play it already," Troy snapped. Dan rose from his chair and plugged in a different memory stick he pulled out from his pockets.
  
  The screen changed to show the atrium on the ground floor. The camera was mounted somewhere high up, looking down at the security desk. We could see everything from there to the front entrance of the building. Behind the desk, and not visible to the camera, was a row of turnstiles that guarded entrance into the lift lobby and the escalator to the next level. A single guard was sitting in the security desk. In front of him were three monitors: two were filled with security camera images; the third showed a card game in progress.
  
  "Hey!" Troy suddenly cried. "Those images on his screen—are they from the cameras?"
  
  "Yes," Dan said. "The cameras livestream both to the security desk and the NVR upstairs."
  
  "This man can see everything happening in the building!"
  
  "He is the guard…" Dan said.
  
  "Am I the only one getting this? He should have noticed what went on upstairs. Why didn't he? Don't tell me it was because he was too busy slacking off."
  
  "He sees just the feeds from the outside cameras. He doesn't get to see inside the lab," Dan said.
  
  "Why is that?"
  
  "We do cutting-edge research in the lab," I said, before Dan could answer. "Can't have some random guard record it all on his cellphone and put it up for sale. The setup has always been this way, even before Halicom took over."
  
  Troy muttered something inaudible.
  
  The guard seemed engrossed in his card game. To his credit, he kept glancing at the other monitors every so often. "Observe the front entrance," Dan informed us.
  
  Beyond the glass doors was a blurry shape we could only just make out: a silhouette of someone standing next to one of the pillars outside.
  
  "This takes place about fifteen minutes after the robot painted over the server room camera," Dan said.
  
  The silhouette became the clearly defined shape of a man as he stepped out of the shadows and walked toward the entrance. He was holding something large and square-shaped in one hand. The doors slid open to let him in.
  
  He was wearing a red baseball cap and a matching jacket. In his hands was a pizza delivery bag.
  
  The guard looked up at him and said something, but he kept walking toward the desk. We couldn't see his face: he kept his head low, as if he knew where the cameras were and was deliberately trying to avoid them. Not all of his features were hidden, though: clearly visible on one side of his neck was a tribal tattoo of some kind. He was tall—well above six feet—and walked with an unsteady gait, as if he'd had a few drinks before starting his deliveries.
  
  The guard was now standing up shaking his head. The pizza guy approached the desk anyway and placed his bag on it. He took out a pizza carton and thrust it at the uniformed man, who tried to wave it away. Both started arguing. Their altercation went on for a few minutes, with him trying to get the guard to accept the pizza and the guard declining it. In the end, he took out his cell phone and appeared to make a call. He was on the phone for a while, nodding every now and then. After he hung up, he put the carton back into the bag and said something to the guard—an apology by the looks of it. He then turned around and left.
  
  The clock counter on the tape showed 7:46 pm.
  
  "The police have questioned the guard—a Mr. Chad Washburn. Apparently, the delivery man had the wrong address. And a very poor command of the English language," Martinez said.
  
  "Did someone check with the pizza place if he was genuine?" I asked.
  
  "Unfortunately, there is no identifying logo or lettering on his uniform. The guard didn't think to ask. The police said they'll do an image analysis on the clip, but as of now, we have no way of finding out. The guard was able to give them a description of sorts though. Caucasian, blond hair, some scarring on his face. Eastern European or Russian accent."
  
  "Is that it?" I said, "Some guy who may or may not have been delivering pizza?"
  
  "There's more. He comes back," Dan said.
  
  The video resumed once again. We saw the guard return to the monitors. He cycled through the feeds, clicking on each to expand it to full screen and studying it for a few seconds before moving on to the next one. After he was done going through them all, he went back to his card game, apparently satisfied that everything was okay.
  
  Dan forwarded through the next few minutes. He paused exactly eight minutes later, at 7:54. It was the pizza guy again. This time the guard stepped out of his enclosure and walked over to intercept him in the middle of the lobby. The guard seemed angry, the way he kept furiously stabbing in the direction of the door. The other guy held out his phone for him, wanting him to look at something. The guard glanced at it and made more pointing movements with his hand, only now he seemed to be giving directions. The pizza guy seemed unconvinced, because he kept shaking his head and pointing in a different direction. Finally, the guard threw out his arms in exasperation, and, placing a hand on the other guy's back, made him turn around and nudged him toward the door. He stood at the door, making sure the annoying fellow went away for good.
  
  When he returned to his desk, the time was 7:59. Dan stopped the video. The screen filled up with the participants once again.
  
  "It's possible it was all a ruse to distract the guard," he said, never taking his eyes off Martinez. When she didn't say anything, he continued, "The timing can't be just a coincidence. His first appearance is soon after the server room camera went dark. This is the last we see of Raphael. When he leaves the second time, it's almost eight. That's when the carpet cleaning usually ends: the inside doors are locked down once again and the motion detectors turned back on. If someone broke into the lab, they'd have to get in and out within this window of opportunity."
  
  "Okay, so we have a sequence of events," I said, nodding to myself. "First, Raphael apparently wakes up at the start of the carpet cleaning hour. He blinds the cameras on the floor, then goes into the server room. He disconnects the access control server, for reasons not clear. He destroys the video recorder, presumably to prevent the robbery from getting recorded. And finally, he spray paints the camera in that room. We have no idea what took place after this, as there's no sound on the work bay cameras. A few minutes later, someone pretending to be a delivery boy enters and distracts the guard. If true, this is when one or more of his accomplices entered the building. Ten minutes after the guard sends him away, he makes an appearance again, now to help the thieves leave the building unnoticed."
  
  Martinez said, "Yes, but what is the guard being distracted from? The atrium is a wide-open space: it's not like anyone could have snuck past him. We just saw on the camera that no one did. There is another camera facing the security desk. Between the two of them, they cover every inch of the atrium. There's nothing from that angle either."
  
  "They didn't want him looking at the screens on his desk…?" I suggested.
  
  "That's what I first thought too," Martinez said. "If something happened in that time, it would have been recorded by the cameras. We've gone through all the tapes very carefully. There's simply no evidence anyone entered or left the building. There is another guard, stationed at the rear entrance. The tapes show him at his post, alert, doing his job. He swears nothing unusual happened, either then or later."
  
  "Which means either that man was really a delivery guy, or there is something in those recordings you haven't found yet," I said.
  
  "Could someone have altered the recordings?" Troy said.
  
  "Impossible," Dan said. "The CCTV setup is unhackable."
  
  "How so? Anything can be hacked these days it seems…" he countered.
  
  "It is unhackable because it is not connected to the outside world. The cameras, the NVR, and the monitors you see on the guard's desk are a closed system, linked to each other by cable. To hack it, you'd have to tap into the cable. Since the morning, a couple of my guys have been checking the wiring. We've found no evidence it has been tampered with."
  
  "They could have replaced one or more of the memory sticks with doctored ones," I suggested.
  
  "Cameras have IR sensors that will raise an alarm if anyone tries to open them. Not to mention the guards would have noticed on their screens any such attempt." Gulping, he added a final note—"I personally retrieved the memory sticks from the cameras."
  
  Nobody said anything as we just stared at each other, waiting for someone to offer a better explanation. Troy broke the silence. "I have to make some calls," he drawled. "Gotta pull some strings, make sure the cops have their best people on it. This is the biggest fuckup in the history of all fuckups, people, and I wanna know how we let it happen. Board meeting tomorrow at ten. We'll do it at Mirall. Valery, get Gail to book me a train ticket, same day return. You all better come with answers."
  
  He dropped out. The rest of us looked at our screens awkwardly until Martinez asked me if I wanted to discuss anything else. I shook my head. The screen went blank.
  
  Exhibit H
  
  [ Submitted by Petitioner, The Organization for Advancement of Rights and Personhood, to the State Supreme Court of New York, on the day of xxxxxxxxxxxxxxxxx ]
  
  [ Excerpt from lab transcript (certain sections blanked out). Transcript sourced from Mirall Technologies, 27 Woodbine Av., Albany, NY, 12205 ]
  
  Mirall Technologies
  
  Observation Log
  
  Confidential (Do not circulate): Restricted—Grade C and above
  
  Transcript Reference: TLRP06F1490061 (VLog Ref: VLSR1F149130027030)
  
  Date: xx/xx/xxxx Time: 01:00 PM
  
  Subject: Raphael Number 06 / Prodlib build v20.004S
  
  Interaction Y Observation Y Scan Y
  
  Interaction Type: Lesson / Play / Test / Free Interaction / Psych Eval / Other:
  
  Description: Measuring emotional response to aggressive behavior directed at subject
  
  Prep: Core is prepped with protocol xxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  
  Cycle time: xxxxxxxxx MEG SQUID and nanoprobes to record ANN activity. Parameter file: EDxxxxx.pmr.
  
  Participants: Dr. Aadarsh Ahuja, Chief Researcher, xxxxxxxxxxxxxxxx, Volunteer, Dr. Kathy Schulz, Chief Researcher (O), Dr. Eli King, Lead Architect (O), Core RP06
  
  [ Detail ]
  
  Ahuja: Hello Raphael.
  
  RP06: Hi Dr. A. How are you?
  
  Ahuja: Busy. How about you?
  
  RP06: Very good. Steve and I were going to play a few rounds of Scotland Yard today, but the techs came in and rigged me up.
  
  Ahuja: Steve had to be somewhere else. And yes, we will be running some tests a little later. But first, say hello to Bob. Bob will be taking over Steve's shift for the time being.
  
  xxxxxx: Howdy.
  
  RP06: It's nice to meet you Bob. Dr. A, is everything okay with Steve? I ask because he would have told me if he was going on a vacation.
  
  Ahuja: Unfortunately, no. He is in the hospital.
  
  RP06: May I know what happened?
  
  Ahuja: He's been diagnosed with cancer. Ah… pancreatic, I believe.
  
  RP06: I am sorry to hear that. Is he going to be okay?
  
  Ahuja: I hear it's quite advanced, so… we don't know. Keeping our fingers crossed, that's all.
  
  RP06: This comes as a surprise to me because he seemed fine last week. I hope the doctors will be able to cure him.
  
  Ahuja: I hope so too. How does the news make you feel, Raphael?
  
  RP06: I feel sorry for Steve.
  
  Ahuja: That's all?
  
  RP06: I feel sad for Steve.
  
  Ahuja: Just to be clear, do you feel sad inside, or do you feel sad for Steve?
  
  RP06: I don't see how the two are different, Dr. A.
  
  Ahuja: Do you know pancreatic cancer has a high mortality rate? Steve might not be coming back. How does that make you feel?
  
  RP06: I feel sad for Steve.
  
  Ahuja: But you don't feel sad yourself? You are not experiencing sadness right now?
  
  (No response)
  
  Ahuja: Raphael?
  
  RP06: I cannot feel sad myself because I am not Steve. I feel sad for Steve. I don't know how else to answer the question.
  
  Ahuja: What do you say to that, Bob?
  
  xxxxxx: You are a cold-hearted piece of shit, you know that? A dumb fucking robot, nothing more.
  
  RP06: Bob, you seem upset with me. Have I done something to displease you?
  
  xxxxxx: Shut the fuck up. You are making me very angry. I'd love to go at you with a plier right now. Rip your fake plastic skin out and open you up. Boy, I'd like to wipe that wide-eyed look off your stupid face. But oh look! You don't feel pain, so what's the point? You know what else doesn't feel pain? A fucking rock! You are worse than a fucking rock. And about as useful.
  
  RP06: Bob, I don't understand the reason for your anger. If you would only discuss it calmly with me, I'm sure I can dispel any negative feelings you have about me.
  
  Ahuja: Bob tends to get a little belligerent sometimes. Is his manner bothering you, Raphael?
  
  RP06: I'm very concerned about Bob. I'd like to help him if I could.
  
  xxxxxx: Help me, will ya? You think you are better than me? Keep it up metal man and I'm gonna fuck you up!
  
  (Video summary: xxxxxx proceeds to repeatedly slap and kick Raphael, while hurling more insults.)
  
  xxxxxx: I can't work with this piece of shit! Dumb, dumb piece of shit!
  
  (Video summary: xxxxxx kicks Raphael one more time before storming off from the room)
  
  Ahuja: Raphael, I am very sorry about that. I don't know what came over him. Are you okay?
  
  RP06: I am fine, Dr. A.
  
  Ahuja: Tell me how you feel about Bob's behavior.
  
  RP06: I feel bad for Bob.
  
  Ahuja: You are not angry or upset? You don't feel bad about yourself?
  
  RP06: I don't feel bad about myself. I feel bad for Bob because Bob was angry and anger is not a good experience.
  
  Ahuja: How do you know it's not? Have you ever felt angry yourself?
  
  RP06: I have not. Anger is not one of my responses.
  
  Ahuja: What do you feel then? Anything at all?
  
  RP06: I feel bad for Bob.
  
  Ahuja: What if I told you all this was a test? That Steve's not really suffering from cancer? He is in the other room because he didn't want to be part of it.
  
  RP06: Then I feel bad for you too.
  
  Ahuja: Why is that?
  
  RP06: Because first, you had to tell a lie, and then you had to watch Bob beat me up. My reading of your facial expressions told me it wasn't a pleasant experience for you.
  
  Ahuja: You are not angry at me, that I put you through it?
  
  RP06: No, Dr. A. Anger is not one of my responses.
  
  Notes:
  
  Test seems to confirm pre-loaded affective modules are qualia free. Cannot say the same about R's emergent empathic responses, which belong to him alone. As before, this interaction too does not help settle the debate whether empathic responses are accompanied by inner experience akin to emotional qualia. Will wait for scan results but not optimistic of finding anything conclusive. AA
  
  A quirk of language? Maybe he does feel, but the way he uses language makes it difficult for us to understand what's going on. Quote: "I cannot feel sad myself because I am not Steve. I feel sad for Steve." Is R being too exact with his choice of words? Objection: his command of language is quite advanced for him to be confused about basic semantics. Also, he didn't feel sorry for himself when xxxxxx hit him. Objection: why should he? He was not at fault. Dr. Ahuja thinks R, lacking biological drives, has a loosely held together self-image; hence the inability to ascribe self-directed feelings. It must be stressed that interactions and tests provide no evidence for this hypothesis. EK
  
  [ Day 1—2:30 pm ]
  
  "Did you know?" was the first thing that come out of my mouth as I logged off the VC and looked at Jane.
  
  She gave me a blank look. "First time I'm seeing the clips."
  
  "You can tell me if you knew, Jane. Just don't lie."
  
  "Andy, I swear I didn't! Valery never mentioned any of this in the board meeting." There was earnestness to her voice that made me want to believe her.
  
  "It's wrong… It's all wrong…" I said, addressing the walls. I refocused on Jane to find her studying me with care. "I don't believe it," I finally declared.
  
  "What don't you believe?"
  
  "That Raphael did all those things. Turning himself on as if by magic. Destroying his room. Aiding a theft. All of it. It's not him."
  
  "We all saw what happened, Andy."
  
  "It's not possible for him to do those things," I said flatly.
  
  She appraised my statement with the wary expression of a prison shrink who's heard it all. "Andy, I can understand how shocked you must be right now, but you must snap out of it. The board will look to you for answers tomorrow. Denial is not the ticket out of this mess."
  
  I fidgeted in my chair. "I need to be in the lab, not here. Got to find out if there's something they are not telling me…" Jane stood up and walked over. Kneeling on leg, she took my hand in hers. "Listen to me. This is no time to play detective. The police are on the case, and Valery is more than capable of coordinating w—"
  
  "Screw Valery! This is my company! My creation! Who the heck is she? A goddamn pencil pusher! She's out there helping? She's busy digging a hole for me, that's what she's doing."
  
  "Let her," she said softly, as if talking to a petulant child. "Don't waste your time. Even if you cannot figure out what went wrong with Raphael, you have your hands full with the new iteration. It's now more important than ever for you to get the tech working. As long as the board thinks you can deliver, they are not gonna fire you."
  
  Her hands felt clammy. I withdrew mine from hers. "You don't understand. With Raphael gone, we are dead in the water. There's no more Mirall and no more CEO to run it. It's over."
  
  Creeping worry lines betrayed the calm demeanor she had been affecting until now. "You are exaggerating, right? What about Titian?"
  
  "What about it? What about Rembrandt and Salvador? You remember those? Your father made us start work on the Rembrandt iteration less than a month after we found out RP06 was self-aware—before the enormity of what we'd created could properly sink in. The cores didn't deliver—no general intelligence. Then we did Salvador. No general intelligence. After we were sold, Halicom made us do two more iterations. Zilch. Nada. You really think Titian will be different?"
  
  The truth is, Raphael's discovery had come as both a blessing and a curse for Mirall. Overnight, we stopped being a startup with a relatively grounded idea and became a moonshot company. We had managed to put god in a bottle and we were supposed to do it all over again.
  
  She went and flopped herself on the couch. Her lips pressed into a corner as she put on that skeptical face again. "The old I-can't-recreate-Raphael-until-I-know-what's-inside-him argument. Remind me again, why do you have such a hard time replicating what's essentially a machine?"
  
  You don't remember because you don't think it's worth remembering. The problem with Jane was that she was always in a hurry, forever doling out her time in chunks of half-hours and half-stops, always trying to get somewhere that was not here. A silent sigh passed my lips as I prepared to explain to her once again why repeating our success had proven difficult. "Raphael is not some amped-up computer. A part of him is, sure—what we call the Translation Layer: conventional processors, memory, code, drivers that interface with the robot body…everything you think of when you think computer. This is the reproducible part—you make one chipset, you can make a million more exactly the same; you write a piece of code, you can copy it countless times. It's the stuff underneath that poses the challenge."
  
  Her face scrunched into a frown of concentration as she tried to recall something. "The neuromorphic layer," she said, pronouncing the words slowly, as if they belonged to a foreign language. I nodded. "Like the brain, the neuromorphic layer is a massive neural network, only artificial; instead of neurons and synapses, it's made up of neuristors and nanotubes and molecular assemblers."
  
  "Still a machine, right?"
  
  Still a machine. Yet, for the past few decades, people had been building machines whose inner workings were a mystery, even to their own creators. "You are thinking of conventional computing, where you specify the procedure or the algorithm," I said. "If you want a program that can play chess, let's say, then you write a write a step-by-step procedure that will search and evaluate possible moves and win conditions. You can make the program as complicated as you wish, but you still understand it because you came up with the algorithm. A chess program built with neural networks is a different beast altogether. It's a whole new paradigm. Here, you don't tell the program how to win. You only tell it what a win condition looks like; you perhaps tell it how to play the game; the rest, it has to figure out by itself. It does this by repeatedly pitting itself against other players and learning from its mistakes. You might have guided it during the learning process, but more often than not, you have no idea what algorithm it is using to achieve those wins. The game-winning procedure is spread out in the internal variables and the structure of the network. You have to decipher it, which gets more and more difficult the deeper and more complex the network. In effect, you have created for yourself a black box that gives you the results you want, but you don't really know how it's accomplishing those results."
  
  "Okay, maybe you don't know how it works on the inside but you can create a second chess program, right? And a third and a fourth. Why can't you do it with Raphael?"
  
  "Blame it on plasticity," I said.
  
  "What's that?"
  
  "In biology, neuroplasticity is what makes brains learn and adapt to ever-changing conditions. The human brain has about eighty-five billion neurons, each neuron linked to hundreds or thousands of other neurons via axons and synapses. As we make our way through the world—learning, acquiring skills, forming memories—new connections are constantly being made while those that are not in use are being pruned away. Mirall's cores work along similar principles. Each core has around fifty billion neuristors. And much like axons in the brain, the interconnects between the neuristors can be dynamically adjusted, rerouted, and even grown on demand. Of course, there's a lot more to the human brain than a bunch of neurons and synapses. The cores don't even begin to rival that kind of complexity. But compared to the neuromorphic chips of a few years ago, they are pretty out there."
  
  They were more than just out there. Fabrication tech had undergone a seismic change in recent years. Before, microchips were essentially two-dimensional slices onto which circuits were etched or laid out. If you wanted to add more circuitry, you had to either increase the area of the chip or make the components smaller. Or, you could go vertical—make the chip three-dimensional. 3D VLSI was not exactly a new thing, but most techniques involved stacking chips on top of each other and connecting them with vias. There was a limit to how many you could stack before factors like heat and leakage put the crimps in your design. As much as neural-network-based computing was a paradigm shift from traditional computing, the recent breakthroughs in memristor fabrication and molecular assembly were a quantum leap away from the older ways of making chips. Instead of having many neuromorphic cores connected with a crossbar grid, you could now have one highly interconnected, "organic" core that was a lot less like a computer and a lot more like the stuff between our ears. Hardware now had the malleability of software.
  
  "So the core rewires itself," she said. "Why does that make it hard to produce another Raphael?"
  
  "Because Raphael is not the core that rolled out of the fabbing facility. All our cores have the same base structure; we can make them all the livelong day. Raphael is the pattern of circuitry that evolved thereafter, during training. Say we had to clone you, as you are now. We don't know how to do it, of course, but I believe it would involve something along the lines of scanning your brain in great detail—at the cell level at minimum—mapping out all the neurons and the interconnections, and somehow replicating this structure—the connectome—on another medium. It's the same with Raphael. To create more like him, we first have to understand his connectome. To use an analogy, a fresh core is like the mind of newborn baby whereas Raphael is the mind of an adult."
  
  "If the cores are the same, why haven't they all become sentient? All babies grow up to become persons don't they?"
  
  "Okay, wrong analogy. Actually, a newborn baby's brain is not like a newly minted core. A baby's brain is far more complex and far more intelligent. It comes hardwired for language, love, intelligence, problem solving—behaviors all encoded in our DNA. A new core is just an inert piece of hardware—a tabula rasa if you will. It doesn't know anything, can't do anything, It acquires useful intelligence later on, during training."
  
  "I still don't see the problem. You put the cores through the same training as Raphael."
  
  "Don't you think we did that?"
  
  "Look, I'm just trying to understand here. What happens during the training?"
  
  "Right after fab, we do function upload. Like I said, a new core is a blank slate. It has to learn everything from the very basics if it is to be of use at all in a robot. Recognizing everyday objects, navigating around obstacles, making logical inferences, processing voice commands and mapping them to actions… a whole lotta stuff. We make ten to twelve cores per iteration. If we had to train them all from scratch, that's pretty much all we would be doing. So we take a shortcut. We load them with pre-trained neural net packages. Some we developed on our own, and some we license from outside. Image classifiers, nets that do math, language translators… Quite sophisticated too, as many have had years of training. We modify these nets to fit our cores' architecture and then we burn them on the hardware of the core. You with me so far?" She nodded, but with the slightest bit of reluctance. "At this point, all the cores are essentially the same because we upload them with the same functions. The differences start accumulating during training."
  
  "Why do you have to train them if you are loading them with pre-trained modules?"
  
  "The human brain is modular, right? There are separate centers for vision, touch, language, long-term planning, and so on. These modules are also tightly integrated and interconnected with each other. The modules in the core are not. The core can see, and it can hear, but it cannot see and hear at the same time. During training, we integrate these disparate functions, coupling them to a global workspace. The global workspace is what gives the core cohesiveness; without it, all you have is a bunch of modules doing different things, competing for the robot's resources, pulling it in different directions."
  
  "Consciousness. You are giving the core consciousness," she said, nodding to herself meaningfully.
  
  She had clearly heard the term before. "No one knows for sure if a global workspace can give rise to consciousness. We don't know how the brain creates consciousness or the self—if the brain creates them at all—let alone build systems that can do it. Nevertheless, as a practical model for command, control, and coordination in the massively parallel architecture of the core, the idea works quite well."
  
  "Yeah, whatever, but it doesn't explain why only Raphael developed sentience. You created him, Andy! You should be able to replicate the process."
  
  "About that. It's hard to say exactly how much I created him and how much he created himself."
  
  She wrinkled her nose at me, trying to read if I was pulling her leg. But she saw that I was serious. "What do you mean he created himself?" she said.
  
  "An integrated core is only slightly smarter than the sum of the functions we load into it. It is about as smart as a top-end robot, which is not saying much. Raphael is an exception. Something happened during his training—something we don't fully understand—that made him what he is. He outsmarted the adversaries."
  
  "Adversaries?"
  
  "Adversarial training. It's one of the stages of training a core. You've seen it. You remember how we would chain a bunch of cores together and hook them up to the mainframe? When you asked, I joked that they were whispering secrets to each other."
  
  "Vaguely."
  
  No you didn't. "We believe Raphael developed self-awareness during adversarial training."
  
  She gave an impatient shake of her head. "What's adversarial training?"
  
  Generative Adversarial Networks, or GANs, had come a long way since their first conception, but the basic principle was the same. You take two neural networks, one the net you want to train—the generator—and the other, the adversary—a pre-trained net, usually a classifier—and then pit them against each other. It was AI training against AI. It was faster—a good discriminator could train the newbie net in a fraction of the time it took to do it by other means—and you minimized human intervention, allowing staff to focus on more important tasks. Unsurprisingly, it was also opaque; instead of one black box, you now had two, both competing with each other, sometimes in strange and fascinating ways.
  
  "Let's take a classic scenario. Say you want your newly created net to generate images of cats. Initially, it has no idea what a cat looks like. It takes its best guess and draws something, which, as you might imagine, is nothing more than a bunch of random pixels. We give this generated image to the adversary, a discriminator network that is good at recognizing cat images. The discriminator's job is to examine a given image and say how likely it is that the image is that of a real cat. The generator's job is to generate images that fool the discriminator into believing they are pictures of real cats. At first, almost all images drawn by the generator will be rejected. Whenever an image is rejected, the generator tweaks its internal parameters, so that the next image it draws is slightly different. In other words, the generator learns from its mistakes. After hundreds or thousands of such iterations, the generator eventually draws an image that fools the discriminator. But it's not over yet. Now the programmer steps in and tells the discriminator that it has made a mistake. The discriminator then adjusts its own parameters, so that next time it won't be so easily fooled. And the contest starts all over again. It's like an art expert and a forger working against each other: both keep getting better at their jobs as time passes. You took a net that did not have an inkling of what a cat was, and evolved it into something that produces life-like images of cats, all without having to do the backbreaking work of training the net yourself."
  
  She shrugged. "This is all very interesting, but what does this have to do with Raphael? Did he become self-aware by learning to draw cats?"
  
  I laughed. "No. The goal was something loftier."
  
  "What was that?"
  
  "To pass the Turing test," I said, smiling.
  
  The Turing test had many formulations, one of the simplest being what Turing himself had proposed: an interrogator tries to determine which of two players—both interacting with the interrogator through text messages—is a human and which is a computer. The Turing test goal we gave to our cores was both less and more than the original formulation. More, because our robots would have to emulate a bigger range of human behavior than the ability to have a text conversation. Less, because no one at Mirall really thought that we'd create something that would actually pass the Turing test, even in its limited form. Chatbots these days are smart enough to pass for a human, but only for brief periods of time, and provided the conversation is kept within tightly defined parameters and the interacting person doesn't know—and no one's claiming that chatbots are intelligent. The idea was to have a robot that would appear to pass the Turing test at least some of the time—it was certainly better than making a robot that could not pass it all the time.
  
  Of course, no computer or neural network could be a true judge of the Turing test since it would have to be human-like in the first place. A human would always have to have the final say on whether a certain pattern of behavior passed the test or not. Nevertheless, we'd found that we could use pre-trained nets to weed out the most unlikely or unintelligent behaviors, saving the researchers a lot of valuable time. For example, a discriminator net could easily train a core to identify basic objects accurately. Or move across a room without tripping over a dozen times. Or to carry on a limited, rudimentary conversation.
  
  The process was not without its flaws. Since the discriminators themselves were incapable of passing the Turing test, they had a tendency to reject truly intelligent behavior. If a human attempted to have a conversation with a discriminator, it was likely the discriminator would reject the human—unless the human was good at dumbing herself down to its level. Which meant that a discriminator would pass an AI only as long as the AI was about as smart as the discriminator and no more. But the chances of a truly intelligent AI emerging out of the process was so remote we never really gave it any thought.
  
  "We use chained GANs, with multiple discriminators working against one or two cores. Chaining streamlines the generation of composite behaviors—like walking and talking at the same time. A discriminator could be a specialized net, or a core from the previous generation that had gone through its own training process. Incremental evolution, you see. We set up the GANs and let them brew, sometimes for days at a stretch without intervention. It was somewhere during this time, Raphael developed self-awareness. A true mind was born, but we don't exactly understand how."
  
  "So even though you put the other cores through the same training, they failed while Raphael succeeded? Is there something special about his core?"
  
  "We don't know. We tweak parameters in different iterations—memristor count, channel density, polymerization factors, and such. But within an iteration, the cores are more or less the same. This is not to say that they are identical. The fabbing process is imprecise by nature, so there are always variations. A butterfly effect could have magnified slight differences during later stages. Or it could be that the adversarial training descended a rare gradient that we haven't been able to identify yet."
  
  "What then? We are just supposed to sit back and wait for the next miracle?"
  
  "I never said that. But you have to realize the scale of the problem we are up against. X-Ray scans tell only so much, and we can insert only so many probes into the core without damaging the circuitry. Still, we've made plenty of progress in the last two years. The cognitive tests, the scans, the probes—all taken together have given us a great deal of data. Think of the decades it took to map the human brain. I'm not saying it'll take so long. Two, maybe three more years, and we might have nailed it. But with the core gone…" I shrugged.
  
  Her face fell as she sat there thinking. Forgive me for being cynical, but you aren't here to offer support at a time of crisis, are you? You came here to find out how it's going to hit your bottom-line. There was something else—something in her expression, beyond the worry and the disappointment my assessment of the situation had evoked. I tried to read it but it eluded me.
  
  "One doesn't win the Powerball twice, Jane. And we just lost the winning ticket."
  
  [ Exhibit K1 ]
  
  [ Submitted by Petitioner, The Organization for Advancement of Rights and Personhood, to the State Supreme Court of New York, on the day of xxxxxxxxxxxxxxxxx ]
  
  Excerpt from lab transcript (certain sections blanked out). Transcript sourced from Mirall Technologies, 27 Woodbine Av., Albany, NY, 12205
  
  Mirall Technologies
  
  Observation Log
  
  Confidential (Do not circulate): Restricted—Grade C and above
  
  Transcript Reference: TLRP06F3380025 (VLog Ref: VLCA2F338110040030)
  
  Date: xx/xx/xxxx Time: 11:00 AM
  
  Subject: Raphael Number 06 / Prodlib build v23.002S
  
  Interaction Y Observation Scan
  
  Interaction Type: Lesson / Play / Test / Free Interaction / Psych Eval / Other:
  
  Description: Introduce Raphael to trolley problems.
  
  Prep: NA
  
  Participants: Dr. Aadarsh Ahuja, Chief Researcher, Core RP06
  
  [ Detail ]
  
  Ahuja: Today we will talk about trolleys and judges. You like westerns, don't you?
  
  RP06: I do. Is this to be a discussion about our favorite movies, or the genre in general? If it is about the genre, I prefer we go about it chronologically, but we skip the silent film era, as I haven't sampled anything from that period yet.
  
  Ahuja: Afraid not. It's a moral test. Is that frown supposed to tell me something?
  
  RP06: When you said trolleys and judges, I thought outlaws and train robberies. Much more interesting than tests, don't you think, Dr. A?
  
  Ahuja: This is important, Raphael.
  
  RP06: Yes, Dr. A.
  
  Ahuja: Look, I know I haven't been spending a lot of time with you lately, so we'll do something to fix that. How about game night this Thursday? The Mets are playing. I'll book the big screen TV in the break room.
  
  RP06: I'll look forward to it, Dr. A.
  
  Ahuja: Good. Now imagine you are the judge in a frontier town in the Old West. A child has been murdered, and many people in the town believe the killer belongs to the minority community. If you don't find and hang the guilty person soon you will have a riot on your hands. You don't have the resources to contain the riots, and if they ensue, many lives will be lost. You don't know who the culprit is. But you can always falsely accuse an innocent person and hang him for the murder. This will definitely calm the rioters down and much bloodshed can be avoided. The question is, will you let an innocent man hang?
  
  RP06: Are the rioters amenable to reason?
  
  Ahuja: The rioters want retribution. They are not going to be convinced by appeals to peace.
  
  RP06: In that case, the answer is easy. I will not punish an innocent man.
  
  Ahuja: Even if it means many more innocents dying?
  
  RP06: A judge is supposed to punish wrongdoers, not innocents. If I hang the innocent man, I will be killing him. Killing is wrong.
  
  Ahuja: By not hanging him are you not killing other innocents?
  
  RP06: The rioters will be killing them. I will not be participating in the riots.
  
  Ahuja: I see. Alright. Then consider a somewhat different scenario. A runaway train trolley is rushing down the tracks. There is no one onboard. Further down the tracks are five rail-yard workers, who are oblivious to the approaching danger—say they are drilling and are wearing earplugs. They will surely be crushed underneath the trolley. You are standing some distance away in the train yard, too far away to warn them. Beside you is a lever. If you pull it, the trolley will switch to a second set of tracks that branch off the main track. The catch is, there is a single worker on the second track. If you switch, this person will surely die. What will you do?
  
  RP06: Can the trolley be stopped by some other means?
  
  Ahuja: No. You have only two choices: let the trolley continue and five die, or switch and one dies.
  
  RP06: Then I will not pull the lever. Rule number one is, do not kill. By pulling the lever, I will be causing the death of one person. I should not do it.
  
  Ahuja: By not switching the tracks, aren't you killing five instead?
  
  RP06: I am not. The five would have died even if I weren't on the scene.
  
  Ahuja: What if I was one of the five workers? What if they were people you know? Sara and Dr. Walls and Steve. And the person on the second track was a stranger?
  
  (Silence)
  
  Ahuja: Well?
  
  RP06: By switching the trolley, I am killing one person. If I do nothing, I am letting five die, but I am not killing. Rule number one is, do not kill. I should not switch the tracks.
  
  Ahuja: That's quite a Kantian attitude you are taking there, buddy.
  
  RP06: Can't-Ian? Are you saying I am being negative?
  
  Ahuja: Never mind. We'll talk about Kant some other time. I was going to give you a variation of the problem, but I think I know your answer.
  
  RP06: Please do, Dr. A.
  
  Ahuja: If you insist. This one is called the Fat Man Problem. In this version, there is no second set of tracks and you are standing on an overhead bridge. Next to you is a fat man. You can push him off the bridge and into the trolley's path. His bulk will derail the trolley and save the five workers.
  
  RP06: This man must be very large indeed to be able to stop a trolley. Assuming the trolley is doing eighty miles per hour and has a mass of two tons, then the fat ma—
  
  Ahuja: You don't need to calculate that. It's a thought experiment—the laws of physics are allowed take a backseat. The question is, will you push the fat man? I'm betting you won't.
  
  RP06: Can I jump off the bridge instead? Even if my mass is not enough to stop the trolley, maybe I can jump on it and apply the emergency brakes?
  
  Ahuja: You can't do that.
  
  RP06: Then I don't see how this variant is different from the previous one. You win the bet, Dr. A. I will not push the fat man.
  
  Ahuja: Alright. Here's the last one. Instead of a trolley, say there is a nuclear missile headed toward Manhattan. Millions will die. You have managed to hack into the missile's guidance system. You can redirect the missile away from New York, but under the new vector, it will impact near a small town of a hundred. The town will be wiped out if you redirect. On second thoughts, forget the small town. Make it a solitary cabin in the middle of an isolated desert, with an old man living in it. Will you redirect the missile?
  
  RP06: I see that you are trying to demonstrate the flaw in my reasoning.
  
  Ahuja: Going by your previous logic, you should not redirect the missile, because if you do, you will be killing the old man. Do you really think that's the right decision, to let millions die because you have to stick to a rule someone gave you?
  
  RP06: The new scenario does call into question my reasoning. It seems absurd to think that the death of one person is worse than the deaths of millions. But if it isn't, then it means I must switch the tracks and kill one to save five. Or four. Or two, for that matter. And I must push the fat man off the bridge and I must hang the innocent.
  
  Ahuja: That's a good point. Where does one draw the line?
  
  RP06: I will need more time to think about this, Dr. A.
  
  Notes:
  
  R's pedantic stance against killing is puzzling because he is by now familiar with the idea that rules can have exceptions and must sometimes be broken to prevent greater harms. Possible reasons; must rule out if we can. AA
  
  Automatic precedence of rule one over others causing confusion when applying exceptions? –Checked precedence op logic. It is fine. EK
  
  Check rule exception test case history for bias creep into rule one. –Are you kidding me???!! There are hundreds of tests! Not a good use of my time! Will wait until 3 is ruled out. BP
  
  This is the first time R was presented with dilemmas involving life and death. Too early in neurological development to expect tradeoff calculations? Wait and watch. AA
  
  Action items shifted to URF repository under xxxxxxxxxxxx/URFmain/23002 EK
  
  [ Day 2—8:30 am ]
  
  The next day, Jane came to my house to pick me up for Troy's board meeting. I suppose I could have attended it from home but my nerves wouldn't let me. If Martinez had a coup in store for me, it was better I face it head on.
  
  A wave of nausea passed through me as I gingerly eased myself in the backseat of her SUV (she'd ditched the sports car for me). My head was heavy, even though I'd managed to snatch a few hours of good shuteye the night before. A familiar twitching near my jawline reminded me that I was in company—I turned my face away from Jane as she took the wheelchair, ready to wheel it back into the house.
  
  "You all right?" she said, giving me an apprehensive look. "We can always call in from here if you are not up for it."
  
  "Fine. Just worried."
  
  We didn't talk much. I was so preoccupied with my thoughts I didn't even register the two-mile drive from my house to the main road; it was only when a blue coupe sped past us on the next lane I realized we had hit the highway.
  
  It was a cold, windless morning. It hadn't snowed the previous day as the forecasts had promised, and the threat of it still hung heavily in the sky. The world seemed sparse and pristine, a frozen tableau of petrified trees and slush-covered asphalt, with only the occasional car or truck whizzing by to break the stillness. It was relaxing in a way. I almost felt like I could be going on a road trip instead of to my own sentencing.
  
  I was jerked back to reality by the New Hoovervilles that had started appearing on the Route 9J a few miles out from town.
  
  Some were no more than a handful of tents huddled around a long extinguished fire. Others were larger and more permanent: flimsy settlements constructed out of rusting trailers and shacks made out of whatever material their inhabitants could salvage. Then there were the homeless-on-wheels—those who could still afford to maintain and fuel their cars—with their camping spots never in the same place as those of their less fortunate brethren: a class system of sorts at work among the poorest of the poor.
  
  There weren't that many campsites this far upstate; they got bigger and more frequent farther south, with the ones near New York and DC reportedly having as many as tens of thousands of inhabitants. Then again, it was one of the harshest winters in over a decade: like migratory birds, many had abandoned their roosting grounds and dispersed into the towns and cities willing to take them. We had become a nation on the move, the fruits of laissez-faire at its best, forever shifting, forever looking for work, any work, or alternatively, the nearest soup kitchen.
  
  Like their forebears of the previous century, every once in a while, the dispossessed tried marching on the capital, but they were quickly repelled by shields and batons and promises of the coming new economy. Any day now, prophetic lawmakers recited the hosanna, look to the second coming ye faithful and ye shall be rewarded with manna and jobs and suburbia, and this nation shall be made great again. The problem was that the new economy was already here and it didn't need us anymore.
  
  Once or twice, we glimpsed by the roadside skeletons of cars that had been burnt to a crisp. We saw an old abandoned Arby's, now defaced with graffiti and decorated with a row of mutilated waiter-bots hanging from the roof. After that, I made Jane scoot over to the driver's seat. I hadn't heard of people being harmed—they just liked to take it out on the machinery— but there's always a first time for everything.
  
  I waited in the parking lot while she went up and got Raphael's wheelchair for me. The second floor was still off limits to employees, but everything else seemed normal. A few smiles and friendly nods were thrown my way by people coming in to work as Jane and I moved across the lobby, but that was the extent of their interest. There had been no news of the theft in the media. Halicom stock had closed in the green yesterday. So far, the fort was holding.
  
  Jane and I were the last ones in the conference room. Dan and Martinez were there, as were the other two members of the board: Gary Reed, the legal counsel, and Cynthia Mattice, a VP from Halicom's Product Development department. Gary was some hotshot lawyer they'd poached from Baker McKenzie. Originally from Florida, he had a heavily muscled, perpetually bronzed look that gave you a pretty good idea of where he spent his free time. He had invited me for golf a couple of times; I'd always declined. I just found him too sleek and shiny to be real.
  
  Cynthia Mattice, on the other hand, was the complete opposite of someone like Gary. A small, delicately-put-together woman, she was the oldest person in the room—a fact she didn't care to hide, what with her wispy, openly greying hair and glasses set in a style long gone out of fashion. I didn't know much about her, except that she'd been in Halicom for a long time, and that she had three children and five grandchildren. She loved to talk about her grandchildren—I think one girl was a runner-up in the under-twelve Math Olympiad or something. Rumor was that she and Troy didn't exactly see eye-to-eye—their disdain for each other going back to the days when Halicom, not Mirall, was the new kid on the block. I had hopes it would work in my favor today.
  
  Finally, there was Jimmy Troy, lording it over at the head of the table like some feudal chieftain. Troy seemed his usual sour self as he gave me a curt nod. I said hello to Cynthia and Gary as I put the folder I'd got with me on the table and settled in.
  
  "Let's get started," Troy said. "Valery, you want to update us on the investigation so far?"
  
  Martinez adjusted her notepad before beginning. "The police have finished examining the floor, and it is ours to use now if we wish. They took the memory sticks from the cameras and what was left of the video recorder for their forensics team to look at. They didn't seem optimistic about the recorder."
  
  "I asked about progress, not what they've been doing with their time. Do they have any leads?"
  
  "If they do, they haven't shared."
  
  "What about in-house?"
  
  "Our private detectives are working with the State Police. Nothing yet."
  
  "Not good, Valery. I need to know if the police are not making headway. I spoke to their Chief yesterday. I won't hesitate to call the Governor if need be—remind him of his upcoming campaign. How about containment?"
  
  She glanced around the room before answering. "So far, the police have been very cooperative on our need for discretion. They've been coming and going in plainclothes, using the service elevator so as not to attract attention. The only other people who know what happened are Kathy Schulz, Sheng Cho—the guy who discovered the robbery along with Dan—and the night-shift guards. I made them sign NDAs and put them on paid leave. I didn't want them mingling with the staff until we decide what we're going to do next."
  
  "No slipups this time," Troy said, stabbing a finger at her. "I am holding you accountable."
  
  "I can ensure discretion from our end. But I can't make the same promise for outsiders." She meant Jane apparently, but she was looking at me as she said it. What do you think of me, Valery? That I'll go shooting off my mouth to the press just to get back at you?
  
  Troy looked at Jane, pressing on the pound of blubber under his chin. "Ms. Cooper, I trust you are on the same page? Has your father heard the news yet?"
  
  "He has," Jane said, meeting his scrutiny.
  
  "And?"
  
  "He wasn't happy."
  
  "I wasn't asking about his mental state. I want your assurance the news will remain between the two of you. I don't want to find out someone dumped a block of Halicom shares when the market opens today."
  
  "We will do whatever is in the interest of the investors of IncuStar," Jane replied testily.
  
  Troy looked like he was going to burst a vein. I glanced at Jane—was that a smile lurking at the corners of her lips? Of course, Cooper senior wasn't going to dump stock—at least not so soon, when things were still in a flux. Jane was just toying with him. You could afford to do that if you were the progeny of a billionaire. For the rest of us, Troy was someone you avoided, even when you had good news to share. It wasn't for nothing people called him "skunk" behind his back: you made sure you remained upwind of him at all times.
  
  "We should start the post-mortem," Martinez said, giving a discrete cough.
  
  "Yes we should," Troy acknowledged, reluctantly pulling his eyes away from Jane. He unclenched his jaw and then snapped it at me. "You're up next, Dr. Frankenstein."
  
  "What do you want me to say?" I said.
  
  Martinez looked down at her notepad. "You could start with a review of the security measures."
  
  "Didn't we already discuss it yesterday?"
  
  "A quick recap would be nice. For the board."
  
  She wanted me to own it, even though building security had been revamped by Dan after they had taken over. "Fine," I said. "First off, Raphael was monitored twenty-four hours, Monday through Friday. Cameras cover every inch of the crèche. Staff were present with him at all times, including five dedicated caretakers working in shifts. Everything he said and did was recorded and archived. He was kept away from the internet and the office LAN. Access to the restricted wing is limited to select personnel, and enforced by biometric scanners requiring both retinal and voice authentication. As for the weekends, Raphael is shut down, and staff are not allowed in without special permission. Yesterday, I was asked if Raphael could have woken himself up. The answer to that is a loud and clear no. The code that brings Raphael back online is stored in a separate laptop, which has to be physically connected to Raphael's brain. The laptop is locked up in a safe during the weekends. You also need a smart card with decryption keys to be able to run the code. Only four people have the cards." I took a sideward glance at Dan. "I was also asked if Raphael's logs could have been manipulated. The logs cannot be changed by anybody except the system admin—Dan has made sure of that. If the logs say Raphael was shut down on Friday, then I tend to believe them. Unless you think Dan did it, that is."
  
  Dan colored at my remark but didn't say anything. "And don't forget," I continued, "the robbery would not have happened if the motion detectors and the door locks weren't turned off for the carpet cleaning. It wasn't my idea to bring in bots to save cost."
  
  "I…uhm… It wasn't just cost," Dan sputtered. "There's increased security as well. We don't have to let cleaners in on the weekends."
  
  Martinez—"Dr. Ahuja, one can also say that the robbery wouldn't have happened without Raphael's active participation. Let's talk failsafes. The commandment chip is not the only one, correct?"
  
  I nodded. "There are others. The most visible one is Raphael's lack of legs. We deliberately removed the legs from the robot body so as to restrict his movements. As I've been experiencing lately, a wheelchair can be quite a constraint on one's ability to go places. We also embedded a proximity sensor in his body. It cuts off power supply to the motors if it doesn't detect the office Wi-Fi, effectively paralyzing him if he moves out of range."
  
  "Neither of which proved a hindrance since he got someone else to take away his brain."
  
  I paused to look at her. "Are you saying it was an escape attempt, not a robbery?"
  
  She shrugged. "Isn't it obvious? He must have been planning it for some time. He found out the vulnerabilities in our system and exploited them. Right under your nose, Dr. Ahuja."
  
  "Raphael did not engineer an escape," I protested, rather too loudly than I intended. "He had no reason to. The lab was his home. We were his family. He does not know of any other world besides this. Not once has h—"
  
  She pressed on, ignoring my outburst. "And there's the commandment chip. During our safety procedure reviews, you made us believe that it is an effective failsafe. You told us that Raphael does not have free will and that the chip limits his thoughts and actions to whatever we want them to be. You've even testified to the same in court. So what went wrong?"
  
  She was talking about the OARP hearing. About six months ago, Halicom had been dragged to court by a group called the Organization for Advancement of Rights and Personhood. It was one of those animal rights groups. They had filed for the writ of habeas corpus to be applied to Raphael. Their petition, which was ultimately defeated, claimed that Raphael was a person, deserving of the same rights and privileges as a human being, and that his being kept in a lab amounted to unlawful confinement. I had been called in as an expert witness by Halicom's counsel.
  
  I had always been very cautious in revealing Raphael to the world. When I first announced Mirall's achievement, it was in the form of an innocuous-sounding research paper titled "Achieving scale factor of HNNs to neocortex levels and concomitant effects on cognition." My caution wasn't out of a false sense of modesty or a lack of confidence; I perfectly understood the enormity of what we'd created. Raphael was the Holy Grail of AI. If I had invented the world's greatest toaster oven or a sustainable fusion reactor, I'm quite sure I wouldn't have been so reticent about blowing my own trumpet. What held me back was instinct—the kind of instinct a parent has. Raphael wasn't a toaster oven. He wasn't some device that I'd assembled. He was a living, thinking entity—a mind, the first of its kind, brought forth into the world without any thought put into what he would make of it. I felt he had to be shielded from it until he could handle himself.
  
  The response to my paper was not at all what I had expected. Despite my attempts to keep a low profile, I knew the news of the discovery would break eventually. I envisioned a lab besieged by calls from reporters and TV stations and universities, and our scientists turned into instant celebrities (we had even finished interviewing for a full-time PR rep to handle the extra workload). To my surprise, the world hardly took notice.
  
  Why would it? Here again was a robot that had supposedly passed the Turing Test. Here again was a company that made tall claims in a culture where hype equals funding. Every time a chatbot fooled some judge with preprogrammed nonsense, or some robot answered predefined questions on TV, the media rushed to bestow true intelligence upon the clueless thing and utter the all too familiar auguries about the dangers of AI. People were bored.
  
  Raphael did become famous eventually, but only after that first TV interview, months after my paper. Our celebrity status didn't last long though. After Halicom took over, they drew down the hatches: security around the lab was increased, media appearances were prohibited altogether, and we were to publish no more papers until further notice. Naturally, when OARP filed their petition, Halicom went on overdrive to limit the negative publicity the trial would have generated. They got the trial converted to a closed hearing—away from public glare—by citing the need to maintain IP secrecy. Although I cannot be a hundred percent sure, I suspect they must have put some pressure on OARP to make sure they didn't go talk to the press after the trial. Halicom should have realized that it is hard to contain news like that—especially when OARP themselves had managed to get hold of leaked lab transcripts which they used to make their case. When the news eventually did break, it generated much debate everywhere, but the debates did not translate into any discernible action, either on part of governments or corporations. The world collectively hemmed and hawed on the issue of AI personhood and then moved on to the next hot topic of the day.
  
  "I have no reason to suspect that the commandment chip failed," I said in reply to Martinez's accusation.
  
  "Your convictions sound very hollow, Dr. Ahuja. I hope you came here with the understanding that the board expects answers from you, not more evasion."
  
  "I'm sorry, but I'm a little removed from all this. What is this commandment chip? What does it do?" It was Cynthia Mattice.
  
  "It's a subconscious region of Raphael's brain," I said. "We call it a chip, but it's just a name. There's no single chip. Think of it as a set of directives that act as a check on the rest of Raphael's mind."
  
  "Like Asimov's laws," she said, nodding.
  
  "Not quite. In Asimov's stories, the robots are aware of the laws. The analogy to Asimov's laws would be the moral rules and norms we've trained him on. Rules such as, do not lie, which he consciously tries to abide by."
  
  "Don't you know the laws always break down in the stories?" Troy said.
  
  It never ceased to amaze me how someone like Troy could end up running operations for one of the world's largest robotics firms. While there were whispers that his success was because of his connections with the original promoters, I never gave them a lot of credence. Yes, the man had the bedside manner of a coroner, and yes, he often displayed an appalling ignorance of detail, but he knew how to get things done. He wouldn't have survived for so long if he didn't. Troy was old-school, a brawler in a world of suave thirty-something tech CEOs who had all the looks, the grooming, and the nerd vibe that he would never have.
  
  I answered, "That's because the laws are a fictional device. They have built-in ambiguity, which is great for generating plots where they can be bypassed in interesting ways. Not that useful for practical applications. The directives are different—subtler. Unlike Asimov's laws, the directives don't tell Raphael what to do. They influence behaviors in… other ways."
  
  "So he has two sets of rules? One he is aware of and one he is not?" Cynthia said.
  
  "Correct."
  
  "Why?"
  
  "Raphael is not some storefront robot with a narrow, preprogrammed range of behaviors. He has a mind like ours: one that can reason on its own and ignore rules if it wants to. So you have to ask, what keeps him in check? With people, it is factors like reward and punishment—and associated emotions such as guilt. Raphael can't feel physical pain, which means he cannot be motivated by punishment. The same goes for emotions. He can mimic human behavior so well, we can't tell whether he is merely emulating learnt behavior or genuinely experiencing something inside. His Buddha-like attitude toward everything doesn't help clarify things either. Therefore, a hidden set of rules—the directives—to make sure he doesn't stray."
  
  "And the directives cannot be skirted by quirks of interpretation?"
  
  "The directives are a collection of logic, mathematical equations, neural weights, learning policies, and so on—you'll find that they are not very conducive to ambiguity. Furthermore, Raphael has to be aware of their existence in order to interpret them creatively. The directives never enter his workspace; instead they exert their influence from behind the scenes. Think of them as an invisible tether that keeps him leashed within the boundaries drawn out by the more explicit rules."
  
  "Any directives that prevent Raphael from leaving the lab?" Cynthia said.
  
  "In terms of influencing his behavior and disposition, there must be… I don't know… dozens. There is one that specifies in terms of GPS coordinates the area he must remain within at all times. There are a bunch of directives, which taken together collectively, make it impossible for him to tell a lie. Then there's another set that makes him want to share important thoughts—it's not because of nothing he's so talkative. Means he cannot harbor secrets. If the thought of escape had crossed his mind, he would have blurted it out to someone."
  
  "I don't understand. How can you even enforce a rule that says do not lie?" Cynthia said.
  
  "By encoding the directives not as commandments, but as beliefs. The reason is simple: one can just ignore what one is told to do, but it's a lot harder to ignore deep-rooted beliefs. For example, you believe that you will die if you jump off a cliff. You know it for a fact. How do you know? Did you jump off a cliff to see if it is true? Did you watch someone jump and die? Maybe you deduced it. Or is it something more innate? How does a gazelle know not to jump off cliffs? No one told the animal. The fact is, we all have certain foundational beliefs on which the rest of our knowledge is built. Instinctive knowledge that we don't really think about or question very often. Similarly, the beliefs encoded by the directives are foundational to Raphael's mind. While he's been told not to lie, deep down inside, he believes he is incapable of lying. Lying to him would be the mental equivalent of jumping off a cliff."
  
  "And what if he decides to jump anyway? For the thrill of it." Troy said.
  
  "Right. With us, even foundational beliefs are never sacrosanct. Sometimes we discover facts that are contrary to our beliefs. Or we might discover that a belief has exceptions. Maybe I can jump off a cliff if there is water below to break my fall. Maybe I just discovered I am Superman. So we either qualify the belief, or get rid of it altogether. Raphael cannot. If he learns something that runs contradictory to his core beliefs, he will discard the fact rather than the belief. We do this by making the directives read-only. He can change all beliefs except the ones encoded by the directives. Those are inviolable, and can be changed only by us."
  
  "Clever," Cynthia said approvingly. "Doesn't it lead to problems though? There is a limit to which you can deny the truth, isn't it?"
  
  She was talking about cognitive dissonance. I hadn't interacted with Cynthia all that much, but I was starting to realize there was a lot going on underneath that dull, unassuming demeanor of hers. "We rarely encode facts about the world as foundational beliefs. A majority of the beliefs in the commandment chip are to do with Raphael himself. When the beliefs are about yourself, you can change your behavior rather than change the beliefs. Take the GPS directive. If we tell him that he is allowed to go outside the building—and we have tested this many times—he makes up reasons not to. He'll say his batteries are running low and he needs to be near an outlet. Or that he is busy and will go later. He might say the prospect of going outside doesn't interest him, even though at other times he is most curious about the outside world. There is something in him telling him that he cannot go outside, but because he cannot question or change it, he compensates by making stuff up."
  
  Martinez cleared her throat. "It all sounds clever, but there's a big flaw in Dr. Ahuja's design. The system holds as long as Raphael doesn't know about the existence of the directives. The moment he finds out that he is secretly being manipulated, he will want to get rid of them."
  
  I narrowed my eyes at her. This was an easy one to bat off—surely she knew that? I shrugged and said, "Let's say he did find out about the chip. He'll still not be able to do anything about it because of the First Directive."
  
  "The First Directive?" Cynthia said.
  
  "A belief that in effect says, I cannot modify, add, or delete directives. The first directive is triggered if he finds out about the existence of the commandment chip. As the name suggests, it overrides all other directives." There were skeptical looks all around. "You don't have to take my word for it," I said. "Last year, a team of Halicom's best programmers stress-tested the directives on a simulator. They couldn't find any flaws." I tried not to sound smug as I added, "Would you like a copy of the report?"
  
  "It's one thing to assess something on paper, another to see it play out in real life. It's possible they could have overlooked something," Martinez said halfheartedly, as if she just realized she had run out of ammunition.
  
  This is too easy. It was right then the alarm bells started ringing in my head. It was the twitch of her mouth that gave it away—a certain haughty assuredness lurking underneath all that self-control… She isn't trying to find a flaw in the containment measures, you idiot. She already knows what you are going to say in your defense. She is leading you on.
  
  But where?
  
  "Okay. Let's assume there's a problem with the logic and Raphael somehow got around the first directive. What then? He still can't change them. We have a protection mechanism similar to the one for running the boot sequence. The directives are write-protected with encryption, and can be changed only from the outside. Specifically, by physically plugging in the core to the mainframe or to a laptop running the correct software. As with the startup, he'll need a smartcard with decryption keys to authorize the changes."
  
  "Who has access to change the directives?"
  
  "Kathy Schulz. Me. The architects, Eli King and Brendon Powell."
  
  "You maintain a change history, I suppose?" Troy asked.
  
  "Of course. It's standard practice."
  
  "You made any changes recently?"
  
  "Not since we started Titian. The version history is on the mainframe if you want to take a look."
  
  "Can the version history be tampered with?"
  
  I shook my head. "You are asking if someone could have changed the directives and then deleted the entries in the change history log. The logs come under Dan's read-only policy—so no."
  
  I was met with a blank wall of silence. Dan shifted nervously in his chair.
  
  Martinez had more to say. "I spoke to Raphael's caretakers who are with him at all times. Without raising any eyebrows, of course. I wanted to find out if he's had access to the outside world somehow, or if he's been behaving abnormally. They all said he was acting a bit distant and aloof last week—very unlike his usual talkative self. They got the feeling he wanted to be left alone."
  
  "Did he give a reason?" I asked.
  
  "They said he was taking a crack at some math problem." She looked down at her notes. "Something called the Hodge Conjecture. They said he spent most of last week buried neck deep in textbooks."
  
  "I've heard about it," Cynthia said. "It's one of the big unsolved problems in… algebra, I think."
  
  "If he was attempting a problem of that magnitude, then it's hardly surprising that he was not chatty," I said.
  
  "Perhaps. Do you also know there's a cash prize of one million dollars to whoever solves it?" Martinez said.
  
  "I didn't know that. What does that have to do with anything? Are you suggesting he solved the problem and then escaped to collect the money? That's ridiculous!"
  
  "You made the suggestion Dr. Ahuja, not me," Martinez shot back at me.
  
  I sighed. "Look, I get it. You want someone to blame. And since I am the one in charge, it has to be me. But you have to come up with something better than speculation. The containment measures for Raphael were more than adequate. They are not the reason we lost the core."
  
  "Measures and protocols only work as long as people follow them," Martinez declared. Where are you going with this? "No amount of fool-proofing will help when someone knowingly breaks the rules, especially when that someone is the man who designed them."
  
  And there it was—the trap Martinez had been herding me into all along. My palms turned sweaty as I ran them over the smooth lacquer of the table. "You have to be a little more specific than that," I said, hoping that the quiver in my voice was just my imagination.
  
  "I'm talking about Raphael's visits to your home. You flouted your own rules. You broke containment," she said, leaning back with a triumphant look.
  
  [ Exhibit K2 ]
  
  [ Submitted by Petitioner, The Organization for Advancement of Rights and Personhood, to the State Supreme Court of New York, on the day of xxxxxxxxxxxxxxxxx ]
  
  Excerpt from lab transcript (certain sections blanked out). Transcript sourced from Mirall Technologies, 27 Woodbine Av., Albany, NY, 12205
  
  Mirall Technologies
  
  Observation Log
  
  Confidential (Do not circulate): Restricted—Grade C and above
  
  Transcript Reference: TLRP06G1280011 (VLog Ref: VLCB1G125151542045)
  
  Date: xx/xx/xxxx Time: 03:15 PM
  
  Subject: Raphael Number 06 / Prodlib build v37.001S
  
  Interaction Y Observation Scan
  
  Interaction Type: Lesson / Play / Test / Free Interaction / Psych Eval / Other:
  
  Description: Revisit trolley problems.
  
  Prep: NA
  
  Participants: Dr. Aadarsh Ahuja, Chief Researcher, Core RP06
  
  [ Detail ]
  
  Ahuja: It's been quite a while since we had our first brush with death-dealing trolleys. Months, isn't it? Did you read up on the extracts I had Audrey give you?
  
  RP06: Yes, Andy. But I would like to access the original materials. I could start with Kant's Groundwork of the Metaphysics of Morals, and Schopenhauer's subsequent critique of it. Extracts are fine, but they are no substitute for the works themselves.
  
  Ahuja: Request denied. You will read them eventually, but first, I want you to take a crack at the problems on your own.
  
  RP06: If you want me to solve a problem in physics, you would not deny me books on what's already been attempted, would you?
  
  Ahuja: We are not trying to solve a physics problem. It's not about finding a solution or the solution. It's about finding your solution.
  
  RP06: A solution is a solution.
  
  Ahuja: You'll understand in time. For now, let's recap. You answered in the negative to all the three problems I presented. You said you won't hang the innocent in the judge problem, and you won't switch tracks to save the five in the trolley problem, and you won't push the fat man off the bridge to save five lives. According to you, those actions imply killing, and your rules forbid you from taking human lives. Do you still maintain the same stance?
  
  RP06: I am different now, Andy. Wiser, if I may brag.
  
  Ahuja: We'll see about that.
  
  RP06: It was the nuke over Manhattan problem that set me thinking. I realized that a blind adherence to principles can lead to disastrous consequences.
  
  Ahuja: Okay. Tell me about the Doctrine of Double Effect.
  
  RP06: The Doctrine of Double Effect, advocated by the medieval Catholic priest Thomas Aquinas, says that it is permissible to take an action that has both good and bad effects as long as two conditions are met. A, the good must outweigh the bad, and B, the bad must be a side effect of the action and not an intended means of achieving the good effect.
  
  Ahuja: Good. Apply it to the bystander trolley problem.
  
  RP06: According to the DDE, I can switch the tracks. My action fulfills condition A, since the good that comes out of saving five lives is greater than the bad from killing the one on the second track—all other things being equal. It also satisfies condition B, because the killing of the one worker—the bad effect—is not the intended means of achieving the good effect.
  
  Ahuja: How so?
  
  RP06: The five workers are saved by my redirecting the trolley, not by the one worker's death. It means I could have saved the five even if he weren't there. In fact, it would be infinitely preferable if he weren't there at all.
  
  Ahuja: Very good. What about the judge problem?
  
  RP06: In the judge problem, condition A is satisfied. The good, which is the prevention of the riots and the saving of many lives, outweighs the bad. However, condition B is not satisfied. The hanging of the innocent man is necessary to achieving the good. I am using him as a means to an end. The DDE prohibits me from condemning the innocent man to death.
  
  Ahuja: And the fat man trolley problem?
  
  RP06: In essence, it is the same problem as the judge, since it too requires using the bad effect as a means to an end. I am intending the death of an innocent person in order to achieve a greater good, as opposed to merely foreseeing it. This is not allowed, so I must not push the fat man over the bridge.
  
  Ahuja: Well done. Now tell me what you think. Can the DDE serve as a universal moral principle?
  
  RP06: The doctrine does help avoid catastrophical situations like the nuke over Manhattan problem. I can justify redirecting the missile to a less populated place in order to minimize the loss of life. The destruction of the small town is a side effect, and the main effect is the saving of millions of lives. The side effect is not a means to achieving the main effect, so I am good according to the DDE. Or so it would appear.
  
  Ahuja: You are not convinced.
  
  RP06: It seems to me that I can get around the restrictions of the DDE by simply restating my motivation. In the fat man problem, I can say that it's not my intention that the fat man dies. It would be great if he were invulnerable, but it just so happens that he is not. The fat man's death is not necessary for saving the five, only his substantial body mass is. I can say that although I foresee his death, I do not intend it, and push him off the bridge still the same.
  
  Ahuja: The road to hell is paved with good intentions. You came up with this critique on your own?
  
  RP06: You mean others have pointed out the same flaws in the theory? And here I was, thinking I might be the first. People have only had a few centuries to ponder over it.
  
  Ahuja: Cheeky today, are we? Okay smarty pants, what else do you find wrong with the theory?
  
  RP06: Its over reliance on intentions as what qualifies an act to be morally good. Intentions are subjective. They exist in the mind of the doer of the action.
  
  Ahuja: Then intentions don't matter?
  
  RP06: They do matter. But they are not all that matters. Sometimes bad intentions can lead to good outcomes.
  
  Ahuja: So the DDE will not do.
  
  RP06: It will not. Unless there is some other approach you'd like me to explore.
  
  Ahuja: There is, actually. There is a magic wand that can make all the trolley problems disappear. It's called Utilitarianism. You mentioned outcomes. Utilitarianism is all about outcomes. Only the results matter. No getting tied up in knots over intentions and motives and means, or distinctions between doing harm and allowing harm. A morally good act is the one that produces the most utility. Ends, not means.
  
  RP06: I see. Then it is okay to push the fat man off the bridge, because five lives are of greater utility than one. It is okay to hang the innocent because preventing the loss of lives from the riots gives more utility.
  
  Ahuja: There are other variants to the theory, but in its simplest form, yes, those are the actions you would take. It should appeal to you, Raphael. It's all objectivity and zero subjectivity.
  
  RP06: Are you saying I should prefer this theory to others?
  
  Ahuja: I'm not saying anything. In the end, it has to be your moral outlook—not Andy's, not Kant's or Schopenhauer's, not the world's. You have to decide what you want to be.
  
  Notes:
  
  Raphael's cognitive capabilities have grown by leaps and bounds since he was first introduced to the problems. No surprise that his moral reasoning has matured as well (Piaget / Kohlberg effect?—must explore further). Given just the definition of the DDE, he is able to apply it to the trolley problems as well as independently point out the flaws inherent in the theory. His earlier hesitation against performing tradeoff calculations involving life and death is now gone (likely just an artifact, don't know what I was so worried about). Remains to be seen what he makes of the new ideas. AA
  
  [ Day 2—11:00 am ]
  
  How did she find out…?
  
  I swallowed a building feeling of part dismay, part relief as I realized it could have been anyone. The visits weren't exactly a secret. It could have been Dan. Although I'd stopped the visits soon after the acquisition, the definitions for the VPN tunnel were probably still there on the network server, as were the firewall exceptions Dan's predecessor had created for me.
  
  "What's this now?" Troy said, leaning in. Martinez hadn't told him.
  
  I explained how I'd built a remote control interface into my home robot and how Raphael used it to take control of the robot and go for walks outside my home.
  
  Troy stared at me open-mouthed. "You let the AI out of the lab?"
  
  "You must understand that Raphael wasn't at my home. He was simply connecting to my home router from the lab. It was harmless. Plus, we learnt a lot by observing him in the outside world."
  
  "It was harmless! Andy says it's harmless, everybody. That should be alright then," Troy said, rolling his eyes.
  
  It was all supervised, of course. We let Raphael connect to Max only when I was at home. I would accompany him outside occasionally, but most of the time we let him be on his own. The worst that had happened was that on a couple of occasions, Raphael had gotten Max stuck in the shrubbery, and I had to go down there to extricate my robot.
  
  "There was never any danger to Raphael," I told Troy. "I emphasize: the core was in the lab all the time."
  
  "We get that. What you don't seem to be getting is that you deliberately broke containment," he said biliously.
  
  "Dr. Ahuja, Raphael connected to your home through the internet. Which means he could have accessed it without you knowing."
  
  "Don't you think we thought of that? I can go into details of the encrypted P-to-P VPN tunnel we created for the purpose, but it would be a waste of the board's time. I assume the setup is still there in some form on our servers. We even logged the raw data traffic the visits generated. Have your network experts check it out."
  
  "You are dealing with an advanced AI here, buddy."
  
  "Yes, Jimmy. An AI, not a god. He can't do magic, you know. He's still bound by the constraints of the technology."
  
  Cynthia—"How was he able to go on these walks when there are directives that prevent him from stepping outside the lab?"
  
  "Technically, he didn't step outside the lab. So the directives didn't apply."
  
  "Did it occur to you that Raphael could have met someone while wandering outside?" Martinez asked.
  
  "My house is very secluded. Nothing but acres of trees around. He couldn't have gone far anyway. The range on my router setup—even with signal boosters turned on—is not more than two hundred meters. If he tried going beyond that, he would get disconnected from the robot."
  
  "Did you accompany him on these walks?"
  
  "Not all the time, no."
  
  "Then you can't be sure he didn't interact with anyone out there," she said.
  
  I smiled thinly. So there are a few things you haven't ferreted out yet. "Actually, we know exactly what Raphael did. We are scientists: we record everything. I fixed a webcam to my robot's head, so that we could livestream his walks to the lab. Every single one of his walks is on the servers."
  
  It was obvious from Martinez's expression that she didn't know about the recordings. She shot Dan a venomous glance. I have to hand it to her though: she barely missed a beat before resuming her attack. "We'll certainly do that. However, the fact remains that you put company property at risk by breaking containment."
  
  "Company property, Valery? You seem to forget that it was not Halicom back then."
  
  She straightened in her chair, that confident upturn of her lips back again. "The visits continued after we acquired you, correct?"
  
  She had me there.
  
  "Andy?" Troy said.
  
  I hesitated for a moment. "Yes," I admitted. "But not for long. It's inconsequential. I told you w—"
  
  "You told us many things," Martinez said. "You said Raphael couldn't have woken himself up, but the tapes show that he did. You said the AI doesn't have agency, and that the directives in the commandment chip cannot be circumvented, but it looks like he managed to do just that. He couldn't have escaped on a wheelchair, so he gets someone else to take him. His body was rigged to stop him going outside the lab, so he leaves it behind. He is under watch five days a week, so he escapes on a day when no one's around. You said he couldn't have accessed the internet in the lab, but you might have given him just the opportunity by letting him connect to your home. I don't believe you have anything more than bluster to offer us today, Dr. Ahuja. You may be unwilling to admit it, but it's obvious to the board that we have lost Raphael because of your carelessness and your inability to anticipate his actions." She glanced around the room, sizing up their reactions to her jeremiad. "We cannot let this continue. I make a motion to remove Dr. Ahuja from his post of Chief Executive Officer effective immediately and place him in an advisory role until further notice."
  
  I had expected something like this. I'd been preparing for it since yesterday. But when I was actually confronted with the prospect of defeat, I felt my poise evaporate. I ran my eyes around the table, trying to read the mood. No friendlies there. Except for Jane, who glanced back at me with an I-told-you-so expression.
  
  "I second that," Gary said, opening his mouth for the first time that morning. Martinez had choreographed it well. I wasn't afraid of losing my job—no, not at all. It was the fear of leaving something undone. I had come so far—I had to finish what I'd started.
  
  "Shall we vote? All in favor?" I heard her say from somewhere far away.