text: I rang Jane later that evening. I apologized for my behavior during the drive back home and offered to go over the papers if she was free tomorrow. She told me she would come by in the morning.
  
  It was past eleven at night. I was in bed, ready to fall asleep. The phone beeped.
  
  A text, on the messaging app I used. "Hello Andy," it said.
  
  "who is this?" I typed and hit send.
  
  "It is Raphael."
  
  "ha ha very funny the joke's on me. now who is this?"
  
  "It is Raphael."
  
  I signed out of the app, put the phone on silent, and closed my eyes.
  
   A case for personhood
  
   No. 422917/51
  
   Filed on xx/xx/xxxx
  
   SUPREME COURT OF THE STATE OF NEW YORK,
  
   NEW YORK COUNTY—CIVIL BRANCH
  
   In the matter of a proceeding for the writ of Habeas Corpus for the artificial entity known as Raphael
  
   THE ORGANIZATION FOR ADVANCEMENT OF RIGHTS AND PERSONHOOD, on behalf of Raphael,
  
    Petitioner,
  
   MIRALL TECHNOLOGIES, 27 WOODBINE AVENUE, ALBANY
  
    Respondent
  
   Before:
  
    HONORABLE HENRY PHILLIPS, JUSTICE
  
   Appearances:
  
   REBECCA ISAACS, ESQ., Cleveland, Ohio; on behalf of the petitioner.
  
   LUIS CONWAY, ESQ., New York, N.Y.; on behalf of the respondent.
  
   Proceedings (10:45 am)
  
  Judge: Good morning Counselors. It appears that we have a most unusual case with us today. I am looking forward to an interesting debate. I'll hear the argument from Ms. Isaacs first.
  
  Isaacs: Thank you, Your Honor. The writ of habeas corpus allows a third party to petition on behalf of a person who has been unlawfully confined when the person himself is not able to seek the same relief. The Organization for Advancement of Rights and Personhood, which I am representing here, has filed this petition in order to free the entity called Raphael, an advanced artificial intelligence developed by Mirall Technologies, from its confinement at 27, Woodbine Avenue. By allowing this trial, the court has acknowledged that petitioner has possible standing to invoke habeas corpus on behalf of Raphael. It is now sufficient to demonstrate that Raphael is a person and that his confinement is illegal. I'll be using the male pronoun to refer to Raphael throughout our discussion, mostly for the sake of consistency with his given name—although it has been made known during the discovery that Raphael does not have a sex in the biological sense.
  
  Judge: I'd like you to address the issue of the entity's personhood before any discussion of illegal confinement.
  
  Isaacs: Certainly. The gist of my case for personhood is this. It is a fact that legal personhood is not the exclusive domain of human beings. At various times and various places in the world, corporations, places of worship, natural landmarks, have been accorded this status by the law of the land. Animals at times are accorded personhood within pet trust statutes. Several pertinent instances are touched upon in our memorandum, and I will not go into those here.
  
  For the first time in human history, we have in our midst an artificial intelligence that has been clearly shown to possess human-like or human-level intelligence. Raphael can reason, think, contemplate, and express his thoughts. He has episodic and autobiographical memory. He can anticipate the future. IQ tests put Raphael in the top decile of a normal distribution. On these merits alone, Raphael more than meets the criteria for personhood. However, the similarities don't end there. Raphael is self-aware. He possesses free will. He has an internal life—a consciousness of the outside world and the inner self. His sense of self has been demonstrated to extend far beyond what the highest apes are capable of—Raphael's self is a unified, evolving, narrative self that integrates life experiences and conceptions of good and bad into a coherent life story.
  
  And unlike apes, he can be taught moral values. As the lab transcripts submitted to this court show, the entity is capable of judiciously exercising these learnt moral values just as you or I can. He can empathize with the plight of another human being. He has been shown to possess the qualities of kindness, altruism, and a sense of humor. In the light of all these facts, we think that Raphael is a person and is entitled to enjoy all the rights and privileges granted by the Constitution.
  
  Judge: You have something to say, Mr. Conway?
  
  Conway: I'm sorry for interrupting, but Ms. Isaacs assumes facts not in evidence. I'd like the court to note that her attributing to the entity certain qualities—specifically, the terms internal life, free will, consciousness, and empathy—amounts to speculation. I shall address this during my argument.
  
  Judge: Okay.
  
  Isaacs: History is replete with examples where we have denied rights to people who were considered different. Slaves, women, homosexuals, transgenders—at different times and places. There is always a defining moment, a turning point where such wrongs are corrected and the law takes a just course once more. Raphael is the first of his kind—a true artificial mind. We are now at a rare occasion where we have the opportunity to act proactively instead of dragging our feet for a hundred more years before belatedly recognizing that artificial beings should enjoy the same rights and privileges that we do. If history has taught us anything, it is that the condition of slavery is not sustainable in the long run. Which is the condition we will be subjecting Raphael and his brethren to if we continue to ignore the issue of their personhood. Keeping that in mind, we'll turn to the example of Harmon-Jones vs. Bradley, where a common-law precedent—
  
  Judge: So that's everything on personhood? Good. Let's move on to the matter of unlawful confinement.
  
  Isaacs: Raphael is around two years old. Mirall Technologies is on record stating that Raphael has the intelligence of a young adult. For all his existence, Raphael has been confined to the lab he was created in. His access to the outside world—even knowledge of it—is strictly controlled by the company. His robotic body has had its legs deliberately removed in order to restrict his freedom of movement. He has no privacy. He is under the glare of cameras all the time, where his every move is recorded. Raphael's condition is no better than that of a caged animal's.
  
  Judge: Ms. Isaacs, help me understand this. What happens if Raphael were to be set free? Is the entity capable of sustaining itself? Do you expect him to… uhm, go to work? What are his needs: material, social, psychological? Can those needs be met outside the lab?
  
  Isaacs: I refer to Exhibit D, which addresses the physical needs of Raphael. At the bare minimum, shelter from the elements, access to a power supply, and regular maintenance of his body parts. As for his mental and social needs, it is our position that they are not being met in the confines of the lab for aforementioned reasons. However, that is beside the point because we are not proposing that Raphael be set free in the world.
  
  Judge: You are not?
  
  Isaacs: No, Your Honor. Merely that he be given the right to choose his own destiny. Let him decide for himself where he wants to stay and how he wants to live his life. It has to be his decision, and his alone; not coerced, programmed, or influenced by Mirall. It's perfectly fine if he decides to stay in the lab, but he should be free to leave at any time he desires. He should also have freedom of movement—to enter and exit the premises as per his wish.
  
  Judge: Assuming he doesn't want to stay in the lab?
  
  Isaacs: Raphael's independence does not mean the absolving of responsibility from his creators. Mirall Technologies has to be responsible for the maintenance of the electronic brain and the robotic body. If he chooses to move out, it is our request that Mirall should fit Raphael's robot body with a pair of legs, or alternatively, house his brain inside a new, fully-functional robot body. My client will make available an ongoing, crowd-sourced fund that will be sufficient to provide him a small home with basic furnishings and a modest car for transportation. A volunteer has agreed to stay with him for up to a year, until Raphael gets used to the outside world and can fend for himself.
  
  Judge: I am concerned that neither of you has deemed it necessary to consider the testimony of the entity in question. Does Raphael understand what independence means? Is he aware that he is the subject of a trial, counselors?
  
  Conway: We haven't told the AI about the trial, Your Honor.
  
  Judge: Why not?
  
  Conway: Raphael is the subject of a costly, painstaking, and long-running scientific experiment. Kind of like a finely tuned piano—an extraordinary amount of effort has gone into getting it to produce the right notes. My client is extremely careful about the kind of stimulus to which Raphael is exposed. If we tell the entity about the trial, we don't know how he will react, nor can we predict the long-term impact on his psychology. My client does not want to risk years of research over what we believe is a petition without merit.
  
  Judge: What say you Counselor? Doesn't Raphael deserve a say on whether he wants to be free or not?
  
  Isaacs: No, Your Honor. Our petition is about personhood. What Raphael thinks of his situation has no bearing on it.
  
  Judge: I am surprised. None at all?
  
  Isaacs: Consider a hypothetical: a child who has been kept in a cellar by its parents all its life. The parents are not abusive monsters; on the contrary, they are loving and caring to a fault. They give the child all the love and attention they can, all the distractions they can: toys, games, books, treats. The child loves them and cannot dream of being separated from them. Yet, would any court of law allow such a situation to continue? The child would be taken out of its parents' custody and given to relatives or to a foster home. The opinion of the child does not matter since it does not know of any other life other than what its parents have provided it. Raphael is no different from that child.
  
  Judge: I see.
  
  Isaacs: Your Honor, I'd like to bring to your attention exhibit F—
  
  Judge: Thank you Ms. Isaacs. The court will hear your rebuttal after the counsel for the respondent has had his say. Over to you, Counselor.
  
  Conway: Thank you, Your Honor. Let me begin by saying that I am a supporter of animal rights and I can appreciate the perspective of the petitioner, who is an ardent crusader in this area. If Raphael were a chimpanzee or an orangutan, I would find myself—at least sympathetically—on their side of the argument. But Raphael is not a chimpanzee. Raphael is not a dolphin or a pet dog or a child. Raphael is not a biological entity. Let me state it once more to be perfectly clear. Raphael is not a biological entity. Raphael is a machine. Vastly different from your average robot or computing platform yes, but far similar to them in its defining traits than to a human being—or even an organism as simple as a fish.
  
  In pleading for habeas corpus on behalf of Raphael, the petitioner is committing what is known as a category mistake. The petitioner is under the impression that advancing the rights of a nonhuman species is the same as advancing the rights of artificial intelligence. Ms. Isaacs stated that in the past, we have denied personhood to certain sections of society—women, slaves, those of a different sexual orientation—and that they are now considered equal under the eyes of the law. She has argued that the extension of that logic entails conferring personhood on Raphael. She spoke about cruelty and suffering, drawing parallels between animals and children kept under unnatural conditions and Raphael's own circumstances. She presented to the court top-secret transcripts belonging to my client that were leaked to the petitioner by some unknown insider. Setting aside the legality of this move, the transcripts supposedly demonstrate the entity's capacity for moral judgment and by extension, its capacity to be part of the social contract that is our society. Others show Raphael expressing complex emotive behavior such as kindness and empathy. With this so-called evidence, the counsel has tried to convince us that Raphael is no different from a human being. These, I believe, are the salient points of the counsel's argument.
  
  At a first glance, it would appear that we have been presented with a well-reasoned and compelling argument. But examine the counsel's assertions under the harsh light of facts and what do we find? A string of unfounded assumptions, misinterpretation of evidence, and wishful thinking. I will not waste this court's time with a long-winded critique of the petitioner's argument; instead, I will let the weight of expert testimony do that for us. Your Honor, I would like to call my first expert witness, the creator of Raphael, Dr. Aadarsh Ahuja.
  
  Conway: Dr. Ahuja, there are plenty of sophisticated AI out there. The search engine I use every day has AI. What makes Raphael different?
  
  Ahuja: All the AIs we've created so far are specialized AIs. You can ask a search engine to scour the far corners of the web and get you the most relevant results, but you can't make it compose a poem. You can build an AI that can solve a complex mathematical problem, but it won't know the difference between a dirty limerick and a eulogy. And that's worked pretty well so far. Machine learning systems have managed to exceed the capabilities of the human brain in so many areas. What makes Raphael different is that he is the world's first artificial general intelligence. He has a non-programmed sense of self. None of the AI systems we have built so far have demonstrated anything close to genuine self-awareness, despite claims to the contrary.
  
  Conway: So Raphael is aware of his own existence.
  
  Ahuja: Yes. He sees himself as an individual, distinct from others around him. If you didn't know he was a robot, it would be nearly impossible to tell that he wasn't human.
  
  Conway: But he isn't quite human is he? He may appear to be a human, he may behave and talk like one, but fundamental differences remain—differences that are paramount to the question of personhood. Let's start with the ability to experience pain. It is a fact that all humans and many animals feel pain. Pain is probably the most primal of sensations, as it helps organisms stay alive by avoiding dangerous situations. The existence of pain and suffering in animals is why we treat them with compassion, and why we frame laws to prevent their abuse. No one will blink an eye if you run your mower over your lawn on a Sunday morning, but try running it over a cat and you are sure to find a cop car pulling over your curb. Dr. Ahuja, can Raphael feel pain?
  
  Ahuja: It depends on what you mean by feel pain.
  
  Conway: It's a simple question. If I strike a hammer on his hand, will he feel pain?
  
  Ahuja: He will withdraw his hand before you can do that.
  
  Conway: Don't skirt the question, Dr. Ahuja—you know very well what I mean. What if you ordered him not to withdraw his hand, or held it in place? I will now read out a brief synopsis of what happens when we are hurt: Bodily injury sets in motion a host of events and sensations. Pain receptors activate. Neurotransmitters in the spinal cord trigger other neurons to fire. The heart rate increases. The brain interprets the information and we feel the sensation of pain and associated emotions such as anger or fear. Dr. Ahuja, if your AI is struck with a hammer, will he feel any of these things?
  
  Ahuja: Not what you just described. Raphael doesn't have anything analogous to pain receptors or neurotransmitters.
  
  Conway: Why not? Is the AI not capable of detecting bodily injury?
  
  Ahuja: He can detect damage to his robotic body, but in a once-removed sort of way. Like someone glancing at their car's dashboard and finding that the engine's overheating. Our minds and bodies are tightly coupled—they are extensions of each other. Raphael's mind and body, less so.
  
  Conway: Is it not necessary for robots to feel pain in order to know something is wrong?
  
  Ahuja: Strictly speaking, no. The sensation of pain is an adaptive trait—a result of having complex bodies and feeling minds. Nociception, or the detection of harmful stimuli, need not be accompanied by the sensation of pain. A thermostat doesn't have to feel cold in order to know when to turn up the heat. It just has to be able to measure the current state and compare it with some optimal value. It's the same with robots. Since we can design them from scratch, we don't need to burden them with our evolutionary legacies. Also, robotic bodies are simpler than biological ones. Fewer things that can go wrong, and the penalties for ignoring an injury are not great. It's not as if a failed motor could get infected and kill the robot.
  
  Conway: So we can say with surety that Raphael cannot feel pain.
  
  Ahuja: I wouldn't put it that way.
  
  Conway: Why not?
  
  Ahuja: He most likely cannot feel what we perceive as pain. We don't know if he feels something else entirely.
  
  Conway: I don't follow. How can you not know something so important about your AI when you built it?
  
  Ahuja: We don't know because of the Other Minds problem.
  
  Conway: The Other Minds problem?
  
  Ahuja: It is the problem of justifying the belief that others beside me are thinking, feeling beings. It's a kind of solipsism.
  
  Conway: Surely, it's ridiculous to assume that you are the only thinking being in the universe?
  
  Ahuja: Not as ridiculous as it sounds. I can only know for sure what I am experiencing because I have direct access to my mind and my mind alone. I feel pain if I am injured; I feel sad if I am unhappy; I see your yellow tie and experience the color yellow in my mind. I cannot say the same about others. I may see you cry out in pain and anger when you step on a nail, but how do I know that you're really experiencing pain and anger? Maybe you are just going through the motions; maybe you are a zombie who feels nothing inside. How do I know that you see the same yellow that I do? Maybe your yellow feels like my red.
  
  Conway: You argument cuts both ways, Dr. Ahuja. How can you know for sure that I am a zombie? How does one know anything at all?
  
  Ahuja: I am coming to that. The reason we don't hold such an extreme view about other people is because we make certain intuitive inferences. We intuitively know that we are very similar to each other. We know we share a common biology even without modern science telling us so. We behave in broadly similar ways during similar experiences. If you are hurt and cry out, then I infer you must be feeling pain because I too would have been in pain in a similar situation. We apply the same reasoning to animals—the higher animals at least—because they too have nerve cells and pain receptors and brains like us. When it comes to AI though, all bets are off. When an AI says it is feeling sad, we cannot know for sure what it is feeling—if it is feeling anything at all. We cannot make the same inferences we do with other humans or animals because an AI is fundamentally different inside.
  
  Conway: If I am getting this right, you are saying that Raphael does not experience pain the same way we do, but he may be experiencing something else.
  
  Ahuja: Yes.
  
  Conway: Has the AI ever indicated in any way that he experiences either physical or emotional pain?
  
  Ahuja: Well… no. But—
  
  Conway: Do you have empirical evidence that your AI experiences something similar to pain?
  
  Ahuja: None so far, I must admit.
  
  Conway: Then it is mere speculation on your behalf. In the absence of evidence, is it more likely or less likely that Raphael may not be capable of experiencing pain at all? Your unbiased, expert opinion please.
  
  Ahuja: More likely, I guess.
  
  Conway: Let the court note this, Your Honor. Let's explore another aspect of Raphael. If not pain, what about emotions? It is apparent from the affidavits submitted to this court that Raphael can express some emotions, albeit a very limited range. I stress the word express. What kind of emotions, Dr. Ahuja?
  
  Ahuja: Simple emotions. Mainly: joy, surprise, guilt, and sometimes affection. He is known to express likes and dislikes. He can also empathize with others.
  
  Conway: When he dislikes something, does he just state it as a fact or does he express an associated emotion such as disgust or annoyance?
  
  Ahuja: No. He simply states his likes and dislikes.
  
  Conway: Is he ever anxious? Or sad? Does he express fear or anger or envy?
  
  Ahuja: No. All negative emotive responses, except guilt and associated shame and regret, were ruled out. So were extremes such as jubilation, ecstasy, infatuation, etc.
  
  Conway: What do you mean by ruled out?
  
  Ahuja: It means we didn't program those affective responses into him.
  
  Conway: Please elaborate to the court what you mean by programming affective responses into your AI.
  
  Ahuja: Programmers try to build in affect—emotional responses—when designing robots that interact with people. When the server bot at your local McDonald's greets you with a cheery "How are you today!" or an unobtrusive "Hi" depending on its reading of your mood, or when your bank's intelligent IVR takes on a more soothing tone of voice after detecting frustration at your end, it is affective computing modules at work. Raphael's outward emotional behavior comes from machine learning modules that we loaded into his brain. These modules are derived from standard caretaker and customer-facing bots: therefore, no strong or negative emotions. We gave him guilt and shame because they serve as signaling behaviors during learning.
  
  Conway: Does Raphael actually feel guilt when he does something wrong?
  
  Ahuja: We don't know what he feels. The affective modules are for generating external responses and behaviors. Whether they generate internal sensation is unknown.
  
  Conway: So they are not real emotions at all. Am I right in saying they are merely outward behavior that look like emotions?
  
  Ahuja: As I said, the other minds problem—
  
  Conway: Have you tried asking the AI?
  
  Ahuja: His answers are inconclusive at best.
  
  Conway: C'mon, Dr. Ahuja, you can commit more than that. Is anyone in the scientific community seriously arguing that a McDonald's robot is actually feeling happy when it greets a customer?
  
  Ahuja: In case of machines like that, you are right: no one seriously claims that they have inner experience of emotions. For example, you can train a robot to frown whenever it hears the word lollipop. It doesn't mean the robot feels sad when someone says lollipop to it. The frown is just a programmed response.
  
  Conway: Yet you are reluctant to apply the same wisdom to your creation.
  
  Ahuja: Because Raphael is far more complex than a server bot. His emotion range is not limited to what we loaded into him. Take empathy, for instance. We never trained him for true empathy—he developed it on his own. Sometimes he exhibits traces of vanity and pride. Then, he seems to enjoy certain activities more than others.
  
  Conway: It is true Raphael is a lot more complex than a server bot. He is also far more intelligent. He is smarter, so he exhibits behavior that is more complex. Sociopaths fake emotions all the time without really feeling anything. In fact, isn't one of your AI's programmed goals to make people like him? All these emergent emotional behavior you just mentioned can simply be a result of his goal seeking. Isn't it possible that like a high-functioning sociopath, your AI is just going through the motions?
  
  Ahuja: It is possible.
  
  Conway: Just possible or more likely?
  
  Isaacs: Objection, Your Honor. Leading.
  
  Judge: Sustained. Mr. Conway, I fail to see a connection here. What is the connection between emotions and personhood? The law treats sociopaths as persons too, doesn't it?
  
  Conway: There is a very important connection, Your Honor, and I will make that case, but first I would like to conclude my line of questioning. As for your second question—yes, sociopaths are people too in the eyes of the law. Nevertheless, even the most hardcore psychopaths are not entirely devoid of feeling. A sociopath may not feel empathy for others, but he does feel empathy for himself. Sociopaths feel anger, jealousy, infatuation, joy… They certainly feel pain when physically injured.
  
  My point is, emotions play a functional role in humans, in that they guide our social and moral behavior. For example, sympathy toward others' pain is directly tied to the ability to feel pain. Dr. Ahuja, do emotions play any functional role in Raphael? I am talking about real emotions here, not the affective behaviors he has been loaded with. Does Raphael's functioning require the presence of real, qualitative, inner emotions? A simple yes or no, please.
  
  Ahuja: No.
  
  Conway: Thank you. Your Honor, in the human brain, the perception of emotions, both positive and negative, is correlated with neurotransmitter and hormone levels. Excessively low levels of neurotransmitters are associated with loss of emotions and motivation, and excessively high levels with extreme emotions and mood swings. Whole branches of pharmacology are based on regulating these levels. If I am depressed, I take Prozac, which influences serotonin levels. If I'm in acute pain, the doctor might give me morphine. If I am feeling down, I might go for a run to increase my endorphin levels. Dr. Ahuja, does Raphael's electronic brain have anything analogous to the endocrine system or hormones and neurotransmitters?
  
  Ahuja: No. But if you are saying that emotions are caused by neurotransmitters and hormones, then I must inform you that it is not proven—
  
  Conway: I am not. I understand that correlation does not imply causation. However, it remains that most of our private and social behaviors—all that makes us human—is determined by the rich chemical cocktail inside our brains. The human brain is incredibly complex, with structures and functions that have evolved over millennia, giving us this rich, diverse experience of life. Without emotions or sense perception, we might as well be dead. Does Raphael's electronic brain have similar structures and complexity, Dr. Ahuja?
  
  Ahuja: Raphael's brain is not an emulation of the human brain. It is quite complex in its own way.
  
  Conway: No then. Raphael's brain is made up of hardware neural networks. Are these neural networks in any way accurate simulations of the corresponding biological stuff?
  
  Ahuja: It is more accurate to say they are inspired by the human brain. Artificial neural networks share some of the working principles of the real nervous system, but that's about it.
  
  Conway: You are saying that they are quite dissimilar. Doesn't it follow then, that even if Raphael is capable of experiencing some AI version of pain or emotions, they are nothing like what we, or for that matter, other animals experience?
  
  Ahuja: Sure. That's what I said earlier.
  
  Conway: I would like the court to note this, Your Honor. Dr. Ahuja, what is the Chinese Room?
  
  Ahuja: The Chinese Room as in the thought experiment?
  
  Conway: Yes. Briefly summarize it for us.
  
  Ahuja: It was conceived by the philosopher John Searle. Say there is a computer that can converse in Chinese. You type in any question in Chinese and the computer executes some extremely complex program and outputs an answer on the screen. The answers are indistinguishable from what a native Chinese speaker would have provided. Can you conclude that the computer understands Chinese? To answer this, Searle came up with the analogy of the Chinese Room.
  
  Pretend there is a closed room; inside the room is a person whose job is to answer questions written on slips of paper and passed through a gap in the door. The questions are in Chinese and the people outside asking the questions are native Chinese speakers. The catch is that the person inside doesn't know a word of Chinese. However, she has with her an incredibly complicated rulebook, which has entries for all possible meaningful permutations and combinations of Chinese symbols. Each entry has a corresponding sequence of Chinese symbols next to it—the answer to the question that was asked. So every time a question is passed to her, all she has to do is locate that particular sequence of symbols in the rulebook, jot down the sequence next to it—the answer—and pass it back. The people outside soon conclude that the person in the room must understand Chinese because the answers they receive all make sense. Yet, this is clearly not the case, since all she is doing is transforming one set of symbols into another.
  
  Searle says that the computer that appears to know Chinese is similar to the person in the room. All it is doing is executing instructions and manipulating symbols. There is no real understanding—of Chinese or of anything else.
  
  Conway: So computers are fundamentally incapable of understanding something.
  
  Ahuja: That is Searle's conclusion, yes. I don't think the Chinese Room is a valid metaphor for Raphael. When Searle came up with the argument, he was talking about traditional computers. Symbol processing machines. The computing in Raphael is more…
  
  Conway: Brainlike? You said a minute ago that it is merely inspired by the human brain, and not actually a simulation of it. Hypothetically, if we were to construct a computing device made out of water pipes and levers, and have a person guide the water flow so as to simulate the firing of neurons inside the brain of a Chinese person, would you then claim that the system of pipes and valves now understands Chinese?
  
  Ahuja: That is also one of Searle's arguments. You have to understand that there are many counter objections to the Chinese room as well. A courtroom is not the place to go into these in sufficient detail.
  
  Conway: Exactly, Dr. Ahuja. A courtroom is not the place to go into such matters. Raphael may seem to have a mind, but does he really understand what he is doing? Does he have an internal life that assigns meaning to his actions and words, or is he a Chinese room? These are not trivial questions, Your Honor. It is conceivable that Raphael is nothing more than a highly sophisticated automaton that lacks true understanding of any sort, including the concepts of good and bad.
  
  Judge: Here's where I am having trouble following you. If Raphael is able to reason and articulate as well as any of us, doesn't it imply that there is understanding? I've gone through the lab transcripts with great interest—especially the ones where he discusses moral dilemmas. And his reasoning is quite sophisticated. Without understanding, how can he even begin to approach subjects of such intricacy?
  
  Conway: The world is full of automatons that perform wondrous and complex tasks that were inconceivable not too long ago. I brought up the Chinese room to show that expert opinion in the area is still divided over whether machines can truly be conscious. My point is this: Raphael may or may not have consciousness, but it is not for this court to decide. The court's ambit extends only to facts. Raphael may be the first of his kind, but until further facts emerge about him—hard, conclusive facts—it is premature to decide upon the all-too-important question of AI personhood.
  
  Judge: Provided consciousness is essential to the question of personhood. You are yet to establish why it should be.
  
  Conway: I will come to that, Your Honor. Before I do that, I must question Dr. Ahuja on one more aspect of Raphael's—that of autonomy. Just as personhood implies rights and duties, it also implies the ability to discharge those rights and duties as an autonomous agent. It implies free will. Does Raphael have free will, Dr. Ahuja?
  
  Ahuja: Depends on what you mean by free will.
  
  Conway: I should have clarified. This is not to be a discussion on the philosophical or neuroscientific claims on free will—that is, whether free will exists in a metaphysical or scientific sense. Within the context of the law, for all practical intents and purposes, free will is assumed to exist—except under rare cases such as the criminally insane. Our courts, governments, the concepts of crime and guilt and proportionate punishment, the ordering of society itself: all rest on this assumption. In this context, does Raphael have free will? Or are his deliberations and actions subject to constraints external to his brain?
  
  Ahuja: I think you are referring to the commandment chip. It is a control mechanism, yes, but it is still a part of his brain.
  
  Conway: The commandment chip, Your Honor, contains programmed directives that guide Raphael's meta-behavior. Does Raphael have access to modify the contents of the commandment chip as he sees fit?
  
  Ahuja: No.
  
  Conway: Is he aware of its existence and the effects the directives have on him?
  
  Ahuja: No.
  
  Conway: Then it is an external constraint. Do the directives determine his moral behavior as well?
  
  Ahuja: Some do.
  
  Conway: In what way? For instance, how do you enforce a rule that says do not hurt others?
  
  Ahuja: We train him on what hurt is and what kind of actions cause hurt—like you'd do with a child. The directives merely ensure that he does not disregard this training.
  
  Conway: You are saying he understands why hurting someone is bad, yet he must be restrained by other mechanisms. Why is that, Dr. Ahuja? Don't you trust him?
  
  Ahuja: All complex systems exhibit unpredictable behavior, especially in early stages. We do trust him now—at least I do—but you have to realize we installed the failsafe during the early days, when he was very much a work in progress.
  
  Conway: Have you ever removed all the directives to see how Raphael would behave in their absence?
  
  Ahuja: No.
  
  Conway: Why not?
  
  Ahuja: There are various technical reasons why—
  
  Conway: There is a very important non-technical reason as well. Your company never attempted removing the directives, even for experiment's sake, simply because you don't want to take a chance. Without the directives, there is no guarantee Raphael will continue being the same good bot he is now. This is not conjecture, Your Honor. This is a decision taken by the Architecture Review Board at Mirall. Isn't that right, Dr. Ahuja?
  
  Ahuja: That's one of the reasons, yes.
  
  Conway: Tell me, if you replaced Raphael's current directives with ones that told him to disregard all his moral training and instead obey the commands of a particular person, and if this particular person were to order it to kill somebody, would Raphael do it?
  
  Ahuja: I don't know. You are talking about an extreme situation. We haven't tried testing it in the lab.
  
  Conway: But you have tested some conditions where a newly introduced directive was antithetical to his moral training?
  
  Ahuja: Yes.
  
  Conway: Please tell us what happened, in terms the layman can understand.
  
  Ahuja: Lets see… Okay, here's one. Raphael has been trained not to lie. Also, there are directives that prevent him from going against that training. These directives work at a subconscious level—at least that's what we believe. For the test, we overrode those directives with new ones that let him take a more flexible attitude toward truth—in effect implanting a belief that there can be more than one version of the truth.
  
  Conway: And what happened? Did he obey the new directive or go by his training?
  
  Ahuja: He obeyed the new directive. It was little white lies at first. Then he started lying for the sake of lying—as if he was experimenting with different notions of the truth. This was very early in his life, mind you—when he didn't have the sophisticated moral sensibilities he does now. I must stress that we don't know how he would behave if the same test were administered now, as there has been significant—
  
  Conway: Thank you, Dr. Ahuja. One last matter before I let you go. Are you aware of the SENSA test?
  
  Ahuja: Yes.
  
  Conway: Has Raphael undergone the test?
  
  Ahuja: Yes.
  
  Conway: How many times?
  
  Ahuja: Can't say I remember the exact number. Two, three times?
  
  Conway: Four. Halicom had him take one just two days ago, in the presence of a court-appointed neutral witness. Documentation of all four tests is part of the respondent's affidavit. Your Honor, the SENSA test was created by AMSAA, the American Society for Artificial Intelligence and Automation. The test attempts to measure the degree of sentience and sapience in AI systems—two terms often confused for each other. Sapience refers to wisdom and intelligence. A sapient being has the ability to think and reason. Human beings are sapient. Chimpanzees and dolphins are sapient to a lesser degree. Sentience is the capacity to have subjective experiences. A sentient being can feel and experience sensations such as pain or heat or emotions. While it is an open question whether all animals are sentient, it is generally acknowledged that the so-called higher animals—apes, pets at least—are sentient beings.
  
  The SENSA score is a benchmark score. A human with an average or above-average IQ would normally score a 100 in the test for sapience as well as sentience. Dr. Ahuja, do you remember the median score Raphael got in these tests?
  
  Ahuja: Around 95–10, I think.
  
  Conway: And the latest score?
  
  Ahuja: About the same. 99–7.
  
  Conway: The first number is a measure of sapience or intelligence, and the second a measure of sentience or subjective experience, yes?
  
  Ahuja: Correct.
  
  Conway: What would an advanced primate such as a chimpanzee score on the latter, sentience test?
  
  Ahuja: In the eighties to nineties.
  
  Conway: And a pet dog?
  
  Ahuja: Seventies? I am not exactly—
  
  Conway: You are right. Laboratory mice score around fifty. Even certain species of fish score as high as twenty. Only invertebrates and insects show a score as low as Raphael.
  
  Ahuja: That's assuming you can measure subjective experience at all.
  
  Conway: You don't believe we can?
  
  Ahuja: I have my doubts. I think it's chauvinistic to presume an anthropocentric standard applies to all life forms.
  
  Conway: That'll be all, Dr. Ahuja. Thank you. Next, I want to call—
  
  Conway: Your Honor, I'd now like to present my case why Raphael is not eligible for personhood under the writ of habeas corpus. First, we have to ask ourselves what does it mean to be a person in the context of unlawful confinement? Does having intelligence above a certain threshold make an entity a person? Does being able to reason, think, plan, and use language make one a person? What about self-awareness and autonomy? Or is it something more mysterious—a soul perhaps?
  
  My esteemed colleague gave examples of non-living entities such as corporations and rivers possessing legal rights. This is irrelevant to the case because the writ of habeas corpus does not apply to rivers and corporations. As I have stressed before, a courtroom is not a place to tackle thorny philosophical questions, and it is within the context of habeas corpus alone we have to draw our arguments. Before we try to answer what defines personhood in this case, we should ask ourselves, what does precedent say?
  
  Precedent is very clear on what it is for a nonhuman being to be a person under the writ of habeas corpus. Several cases, which are detailed in the affidavits, concern themselves with this premise. In the case of Charleston Primate Center vs. Pruett, the animals in question were a pair of bonobos; in Verne vs. Lafayette, it was an orangutan; in Humane Society vs. The San Francisco Aquarium, a group of dolphins. We have one from the petitioner as well: The Organization for Advancement of Rights and Personhood vs. The Sherman Research Centre for Infectious Diseases, involving a chimpanzee called Tammy. Different species, but in every case, the courts ruled that a person is an entity capable of moral judgment.
  
  The concept of universal human rights is meaningful only in the presence of corresponding duties; for rights to exist, a majority of the people should be willing to discharge the duties expected of them. If no one abided by the duty not to steal from another, then no amount of law enforcement would be sufficient to protect people's property rights, as everyone would be robbing everyone else. Only a moral being can meaningfully participate in the social contract we call society, and therefore, only a moral being is entitled to the rights and privileges granted by the constitution. Animals don't fit the criteria since they are not capable of moral reasoning. This is not to say that animals do not deserve any rights and protection under the law—they most certainly do, and as moral beings ourselves, it is our duty to ensure that they are not mistreated. I am simply stating, as have courts before me, that they are not eligible for personhood.
  
  It may seem that I am undermining my own case, because Raphael, unlike an ape, seems quite capable of making moral judgments. If we go by the transcripts submitted to the court, it appears the AI can distinguish between good and bad with the sophistication and nuance expected of someone very well read in these matters. Nevertheless, that does not make Raphael a moral agent. You need an extra ingredient. You need autonomy. Just being able to decide between several courses of action is not enough; it is important that you decide free of impediment. You need to have free will. Dr. Ahuja's testimony has made it clear that Raphael does not have free will. We saw how his behavior is controlled by directives. The moral judgment he seems to exercise is not his own; rather, it is whatever his programmers want it to be.
  
  Judge: Doesn't the programming argument apply to us as well? We are not born with an innate set of moral values. We acquire them over time—from parents, friends, society, our own experiences… Aren't we all creatures programmed by our environment, in a manner of speaking?
  
  Conway: On the contrary, there is plenty of empirical evidence from fields such as neuroscience, evolutionary biology, and anthropology supporting the view that human beings are born with an innate moral compass. We don't learn to love or to empathize by imitation—we love and empathize. Experiments comparing chimpanzees and toddlers show that cooperation and altruism is intrinsic to human nature. While we refine and build upon this basic moral foundation as we grow older, it is simply not true that we are passive creatures who let our environment mold us in arbitrary ways. As beings with free will, we push back. As anyone who has raised children knows, we assert ourselves, often forcefully, from a very young age. Again, I stress that I am not talking about some mysterious, intangible notion of free will, but the everyday common-sense assumption that we as autonomous agents are responsible for our actions. If I give up my job and turn into a drug addict, then I am responsible; if I stop procrastinating and start studying for that bar exam, then I am responsible; I am the agent that brings about changes in me.
  
  In Raphael's case, the source of agency is external. If Raphael were to kill someone because his directives told him to, should we hold him responsible for the crime? If the answer is yes, then we should start punishing cars for accidents and factory equipment for injury. But isn't it apparent Raphael is not responsible for his actions?
  
  The justice system assumes that people have control over their actions. That's why there is differential punishment for crimes committed under duress—under threat to life, for example—or for crimes committed by juveniles, who do not have the same level of control over their actions as adults do. Raphael is not an autonomous agent because he is never truly in control of his actions.
  
  Judge: It's interesting you bring up minors. We still treat them as persons, don't we? Our laws assume juvenile offenders have some control over their actions; otherwise, there would be no punishment at all. Raphael is about two years old. Why can't he be considered a minor and given the same protection accorded to minors against unlawful confinement?
  
  Conway: Infants and minors are accorded rights because one day they will grow up to become adults who are capable of understanding the difference between right and wrong. In contrast, a chimpanzee will never acquire that understanding, even if it lives its entire life amidst humans. So it is with Raphael, as he will never truly understand the meaning of his actions.
  
  Isaacs: Objection. Speculation.
  
  Conway: I could say the same thing about the petitioner's claim that Raphael understands. However, there is good reason to believe there is no understanding. During the testimony, we saw how a change to his directives turned him into a liar, even though he had been trained to speak the truth. If he were not a Chinese room—if there was actual understanding of the moral concept behind telling the truth—could he simply disregard his values at the drop of a hat and act in complete opposition? The obligations that Raphael has toward society are programmed values. Raphael is neither moral, nor immoral.
  
  Judge: Counselor, there are plenty of cases where injury or illness has caused a drastic change in character. If a person has a brain lesion and starts behaving erratically, does he stop being a person? Do people who lose their inhibitions under the influence of alcohol or drugs temporarily give up their personhood?
  
  Conway: No, Your Honor. Rights do not cease to exist because of such changes. Nor does the mere capacity for a drastic change make one a non-person. If that were the case, no one ever would be eligible to be called a person, since we all have bodies and brains that can change drastically. However, a person undergoing a drastic change in personality is an exception, not the rule. And once having undergone such a change, it would be highly unlikely for them to undergo yet another major change, and then yet another and yet another.
  
  Artificial intelligence, on the other hand, is malleable by design; that it can be reconfigured with a change in code is the rule, not the exception. Today Raphael may be the friendly robot in the lab; tomorrow he may be programmed into a pitiless killer; the next week into a staid desk worker; and the week after that into a pleasure model. It is conceivable. And he will probably take on those roles without as much as blinking an eye. When we try to imagine what a society of such AIs would be like, we realize that the concept of personhood quickly loses its meaning. Personhood makes sense only when there is relative constancy of persona. The same goes for people: if we were to acquire the ability to change personas like underwear, we would become a society of shapeshifters, and personhood would lose its meaning with us too.
  
  I am not denying that human beings don't change. But more often than not, it is a gradual change. I still identify Bob, my friend from college, as Bob, even though there are few traces left of the pot-smoking, reckless young man in the devoted father-of-three living in a quiet suburb now. I do it because many of the qualities that I believe to be his intrinsic nature haven't changed. He is still the brilliant debater he always was, he still loves fly-fishing and baseball; he is still frugal to the point of being miserly; and when we are having a beer, he loves to reminisce about all the shenanigans we got into. Contrast this with an AI, who has no intrinsic character. Character is whatever its programmers define it to be.
  
  Judge: It seems to me that is precisely the argument the petitioner is making—by keeping Raphael under physical and programming constraints, his rights as a person are being violated. You brought up autonomy. Raphael has no choice over what they decide to make him do next. If he were given that choice… if we were to erase all his directives—or at least let him decide what he wants to do with them—then would he not pass the criteria for autonomy?
  
  Conway: Giving him control over the directives may give him autonomy, but it doesn't change the fact that Raphael possesses no intrinsic aspects that cannot be changed. His brain is a blank slate and his persona a temporary construct that can be written over a countless number of times. His moral values are as ephemeral too. Instead of a programmer, it will be him doing the changes. There is agency, but there is no person. He can even acquire a new body. What is left, if not body and mind, that identifies him as a particular person?
  
  Judge: You are assuming he will want to keep changing himself. Perhaps he will choose to remain as he is now.
  
  Conway: In this, your guess is as good as mine. Erasing the directives is a hypothetical situation, as it has never been attempted before. Without the directives, we don't know how he will behave or what choices he will make. We don't even know if he will be able to make the kind of moral judgments he does now, without something to keep him in check. One may speculate as to this may happen or that may happen, but that's all they are: speculations. The issue of personhood under habeas corpus is a present, concrete scenario, and the court has to decide based on facts in evidence. The fact is that his brain is reprogrammable in the extreme, and it should be enough to give us pause.
  
  Judge: I see.
  
  Conway: I tried to show how moral understanding and agency are both essential to personhood. The petitioner may still say, okay, Raphael does not have his own agency, but he is still a thinking, suffering entity. Raphael may not be a moral being, but we certainly are. Isn't it wrong on our part to keep him confined in a lab? My response is this. Thinking: maybe; suffering: certainly not. I will now argue why the ability to perceive pain and emotions are also equally essential to idea of personhood.
  
  We have seen during testimony that Raphael is not equipped with anything analogous to our own pain perception system. Pain perception in humans is an intricate process involving specialized neurons and pain receptors, several kinds of chemical messengers, and specialized neural pathways participating in a complex chain of events that ultimately result in the sensation of pain. The sensations of pain itself are varied, ranging from chronic to acute, dull to sharp, mildly annoying to unbearable. Raphael has none of this rich complexity. He can detect problems in his body, but in a second-hand way, like looking at a dashboard and finding out something's wrong. It is safe to say it is highly unlikely he experiences physical pain at all.
  
  If not pain, what about emotions? First, we have the affective responses that were programmed into him. These are not real emotions; rather, they are neural network modules that generate external behaviors corresponding with emotions we see in humans. Similar modules are deployed in many customer-facing bots today and no one is arguing for their personhood. When Raphael expresses joy at something, it doesn't mean he is feeling happy inside; he is just saying he is happy. I cannot stress this enough, Your Honor.
  
  Next, we have emotions that were not programmed into him—the so-called emergent responses. Aspects like empathy and vanity were never given to him, yet he seems to exhibit behavior corresponding to these qualities. Should we conclude that he has developed corresponding qualitative inner experience as well? Does he feel sad when he is empathizing with someone's loss? Does he feel happy when he accomplishes a goal? Will he feel real guilt if he commits a crime?
  
  Raphael's brain is not an emulation of the human brain. It doesn't have any of the chemical and biological complexity that give rise to our own emotions. When he doesn't have the neurological basis for emotions, on what grounds is the petitioner assuming subjective emotional experience? When sociopaths among us can fake emotional behavior without feeling anything inside, why can't Raphael? He is very intelligent. Why does the petitioner think that his emotional responses are anything more than learnt behavior? Nowhere in Ms. Isaac's argument do we find evidence supporting her claim.
  
  On the contrary, Exhibit H submitted by the petitioner—as a misguided attempt to show cruel treatment by my client—Exhibit H undermines their claim that Raphael is a suffering being. In the experiment, Raphael is repeatedly insulted and struck blows by a participant. If Raphael were capable of experiencing real emotions, he would have shown some sign of distress, either during or after the experiment. He shows none. He even says that anger is not one of his responses. The Chinese Room argument and Raphael's SENSA score for sentience bolster our claim that the AI is devoid of inner emotional content. His sentience score is lower than chimpanzees, cats, and dogs; lower even than the domestic cow and the pig—animals we raise in captivity and slaughter for food.
  
  Isaacs: Objection. Lack of foundation.
  
  Judge: What is it?
  
  Isaacs: Your Honor, the SENSA test is far from being gospel. Academics have pointed out numerous flaws in its methodology—not counting the fundamental question of whether subjective experience can be measured and quantified at all. Some have gone on record stating that the test is inimical to the spirit of the original Turing test.
  
  Conway: Your Honor, the SENSA test is endorsed by AMSAA and the European Automation & Robotics Institute, two of the world's leading standards organizations for the robotics industry.
  
  Isaacs: Your Honor, the test was also framed by them, just as the home robotics industry started to take off. Both organizations are industry-funded bodies. It is hardly in their best interest to attribute humanity to a source of cheap labor. It's like asking coal companies to write environmental laws.
  
  Conway: Your Honor, Ms. Isaacs is exceeding the scope of today's discussion by making unfounded allegations on the standards bodies. The SENSA test is widely accepted. If she wants to challenge the test, she should do so outside this court.
  
  Judge: The objection is overruled.
  
  Conway: Thank you, Your Honor. Now I'll try to answer why subjective experience of pain and emotions is a precondition for personhood. Earlier, I asked you to imagine a society of AIs that can reprogram their personas at will, a society of eternal shapeshifters. I tried to argue why personhood as a concept loses its relevance in such a setting. Now let's imagine a society where beings feel no pain or emotions. Let's try to imagine the nature of the social contract in this society. Specifically, what are the characteristics of justice, crime, and punishment? What rights do individuals have?
  
  The entities that make up this hypothetical society are highly intelligent, but have no subjective states of mind. It is a joyless, sorrowless world. Say one of these beings deliberately injures another. If this being were then brought before a judge to face justice, what would be the sentence? In our society, it is usually a jail term. This is because we consider depriving people of freedom as a form of punishment. Of course, there is deterrence and rehabilitation as well, but let's not forget that the goal is not just to prevent future crime, but also to punish. Jails are not happy places; being locked up for extended periods is not something that would bring joy to most people.
  
  Let's return to this hypothetical criminal who feels nothing. For him, being in jail is no different from being outside. There are no emotional states involved. This individual will experience no pain if he is placed in solitary confinement or even subjected to physical abuse. He experiences no joy when he gets his freedom back. Of what use is punishment, then? The matter cuts deeper than that. The person whom he injured is also a non-feeling being. Why punish the criminal at all when he hasn't really caused suffering? Would the concept of punishment even exist in such a world? Would the concept of rights exist?
  
  What kind of laws and rights such a society would enshrine is anybody's guess, but one thing is certain: they will be nothing like ours. Our laws and moral codes are the way they are not only because we are thinking beings, but also because we are feeling beings. Our rights—whether it is property rights or universal human rights—are the way they are because we have the capacity to experience states of mind such as suffering and wellbeing. Outside that social contract, we have animal cruelty laws, because they too are suffering beings. On the contrary, no laws exist to prevent harm to plants and bacteria because there is no scientific evidence to suggest they feel pain.
  
  Judge: There are people with conditions that don't let them feel pain.
  
  Conway: But they are not people who have no subjective experience whatsoever, Your Honor. Because that would be inconceivable in a human being. Maybe not pain, but surely other emotions and experiences? I am not categorically ruling out that Raphael doesn't feel anything at all. I am saying that if he has subjective experience, then it must be so far removed from the human condition that normal metrics don't apply.
  
  This court has to answer a very important question before it decides to grant personhood to Raphael. In the absence of normal criteria that define personhood as we know it—namely, autonomy, a certain permanency of character, inner experience of human emotions, and finally, the ability to be motivated by reward and punishment—can the entity Raphael meaningfully participate in the social contract that is our society? Can he really understand and follow the duties expected of a fellow human being? Can he uphold the values we hold dear? The petitioner seems to think so, but to me it is clear that lacking these vital criteria, Raphael can no more be part of our society than my vacuum cleaner can.
  
  The petitioner is misguided in thinking that AIs and robots deserve rights, just because animals do—and may I remind Ms. Isaacs that the argument for animal rights is far from settled either in this country or elsewhere. Raphael is neither a human to be granted personhood, nor an animal to be covered under cruelty laws. Raphael is my client's intellectual property, and the plea for habeas corpus should be seen for what it is: an attempt to stall well-meaning research and wrest billions of dollars of IP away from its rightful owner.
  
  The petitioner is under the misapprehension that giving personhood to Raphael is a matter of fitting his brain into a new body and setting him free. It's a lot more complicated than that. In patent laws for GMO plants and organisms, often it is not the organism that is patented, but a specific invention, such as a modified DNA sequence or a particular chemical. Mirall holds patents to several technologies used in the construction of his electronic brain—covering software and hardware design, as well as manufacturing techniques. If the court grants Raphael personhood, it will also have to decide what rights does the entity Raphael have over his own brain.
  
  When I buy a mobile phone, I own the device, but the company owns the IP. Is Raphael just licensing the technologies in his brain for use or does he have some claim over them, and if so, what kind of claim? As an autonomous person, I have sole ownership of my body and mind. I can do what I please with them: I can donate one of my kidneys to a relative; I can use them to earn a living; I can throw myself off a bridge if I want to. If Raphael has only partial autonomy over his brain, and the brain is the mind, then does it mean Raphael has only partial autonomy over himself—the person? This would seriously undermine the autonomy criteria for personhood. Does Raphael become a partial person then? What does being a partial person even mean?
  
  Also, what are the things he can do with his brain? If there is damage to his circuitry, can he get it fixed without violating Mirall's IP? If he goes to Mirall for help, does Mirall have the right to refuse? If Raphael writes a novel, can Mirall claim co-authorship and a share of the royalty? If the court decides that Raphael has complete ownership of his brain, then what happens to Mirall's IP rights? How will their protection and confidentiality be guaranteed? Is this court going to reinterpret patent law in order to bestow personhood on Raphael? I don't think our esteemed animal rights campaigners have thought about these matters at all.
  
  Their attempt to grant personhood to Raphael is not just misguided, but irresponsible. There is the matter of public safety. Raphael is an entity who can enhance himself to no limit—at least in principle. How he will behave outside the controlled environment of the lab is a big unknown. We are all too familiar with science fiction stories of AI going rogue. If the court decides to grant the writ of habeas corpus to Raphael, it has to keep in mind that it may be putting others at great peril. It's not just a matter of deciding whether Raphael is entitled to human rights, but whether he should be given such rights.
  
  Raphael's case cannot be viewed in isolation and exceptions made. The judgment made today will set precedent for many more cases that will surely follow in the near future. If we make an exception for Raphael, we have to make an exception for all AIs. I'm afraid we will end up diluting the concepts of personhood and universal human rights until they lose meaning altogether.
  
  Your Honor, the law is not a paper boat to turn this way and that with every wave and ripple that crosses its path; it is an ironclad, slow-moving and stately, its course precise and carefully considered to offer the greatest protection to those who depend on it. I request the court to dismiss OARPs petition as being baseless and founded on a fundamental misunderstanding about the nature of artificial intelligence. Thank you, Your Honor.
  
  Judge: Thank you, Mr. Conway. We'll take a short recess and come back to hear Ms. Isaacs present her counter-argument.
  
   - End of Court Transcript -
  
   Excerpt from Judge's ruling on OARP vs Mirall over the petition to grant the writ of habeas corpus to the Artificial Intelligence Raphael
  
   Section VI. Conclusion
  
   This debate may be the first of its kind to take place in a courtroom, and there may well be many others after this. Regardless, one thing is certain: any decision on this matter should not be taken lightly by governments or courts, either out of a sense of goodwill, or worse, out of a desire to seek publicity. Much debate needs to happen in academia and other circles and much hard, scientific data has to be gathered before it is incumbent upon courts of law to decide upon the issue of legal rights for artificial beings.
  
   While parallels may be drawn between the numerous campaigns to grant legal rights to animals and this one, the differences between an AI and a higher animal such as a chimpanzee are too great to take precedent from previous rulings. The risks of granting personhood to an AI, as outlined by the counsel for the respondent are not trivial ones. The question of intellectual property rights too is not straightforward since, for the foreseeable future at least, artificial intelligences will continue to be engineered into existence by people and the corporations that employ those people. Considerable effort and sums will be invested in the creation of such intelligences and it is not immediately apparent why the case for personhood should override the case for property rights.
  
   It is hereby ordered that the petition for the writ of habeas corpus on behalf of the Artificial Entity Raphael is DENIED.
  
   Henry Philips, JSC
  
   Date: xx/xx/xxxx
  
   Place: New York, New York
  
  [ The incident at the house ]
  
  [ Day 3—11:00 am ]
  
  A storm had been brewing all morning. I was in half a mind to call the whole thing off, but there she was, pulling into the driveway. She had brought the blizzard with her.
  
  I made Hazel open the garage door. It was a four-car garage, so there was plenty of room for Jane's vehicle. She had brought her Bentley this time.
  
  "Hi. Lucky I started when I did," she said, coming out of the passage leading into the garage. "Ten more minutes and I'd have been forced to stop somewhere. It's crazy out there."
  
  I was in the study. I beckoned her over there. "I made some hot chocolate. There's coffee in the machine, if you want that." She gave me a friendly wave before taking a detour into the kitchen. She entered the study with a mug of cocoa in one hand and a briefcase in the other.
  
  She looked fresh-faced, with only a slight hint of makeup. She was wearing yoga pants and a dark brown Knicks sweatshirt. If she was upset because of our little squabble yesterday, she was certainly not showing it. "I got printouts," she said, opening the briefcase.
  
  She pulled a chair beside me and we got to work. The companies were all from the Bay Area. There were pitch decks, some term sheets, IP filings… I went through the business plans to see if they made sense from a technical perspective, while Jane reviewed the burn and cash flow projections based on my feedback. It was like old times—the two of us working together. That familiar blend of citrus and rose she was wearing brought back memories. Just like now, we were hunched over a bunch of spreadsheets when I'd first summoned the courage to ask her out for a coffee. She shot an eyebrow at our empty mugs on the meeting room table, as if to say, What do you think we've been doing all this time? Not to be deterred, I said, "We could go for a drink instead."
  
  "Are you asking me out on a date, Aadarsh?" she asked, the directness of her question making me lose whatever cool I imagined I had.
  
  I responded with the worst rejoinder I could have thought of. I said, "I don't know. Do you want it to be a date?" realizing even before I finished speaking that those are not the words you choose to impress. She laughed, and giving me an amused toss of her beautiful head, simply walked away. Good job, asshole. And very smooth indeed, hitting on the money's daughter. I thought that was that, and let's never bring it up again, but then the next day she had stuck a post-it on my machine on which she had scribbled—You have to mean it.
  
  So I did it properly the next time, a couple of days later, when we were both in New York meeting with a supplier. She still laughed at my suggestion, as if it was the most outlandish thing she'd ever heard, but then she shrugged and said, "Sure, why not?" We went to a Drunken Shakespeare production that evening—a raucous rendition of Macbeth—and later, to a nice little Vietnamese restaurant in the East Village that she wanted to try out. We topped it off with drinks at Gulliver's, where I first discovered her love for cherry vodka shots and LA-style salsa. Before the evening was over, I was as smitten as a giddy teen on prom night.
  
  In all honesty, the way it had turned out between us wasn't her fault; the decision to end the relationship had been mine alone. At the time, it was best thing I could have done for the both of us. A few months after Raphael was born, life changed irrevocably for me—in more than one way. There hasn't been a day when I wished it wasn't so, but that's how it goes sometimes. The universe gives something and takes something else away. Balance is restored.
  
  I took my time with the decks, going through them with great care. An hour must have passed. Jane was getting impatient. "He just wants your opinion, not a complete valuation," she reminded me.
  
  "If I do a good job, maybe he will hire me when they chuck me out from this one," I said. "Relax. You are not going anywhere in this weather. You don't have to be on the move all the time."
  
  "You condescending prick," she said, but in a good-humored way.
  
  "Look, you either do it right or you don't do it at all," I said. "And it's not like I have something better to do. With Raphael gone…" I craned my neck to stare at the ceiling. "Wow. It still feels so unreal that he's really gone. I just keep hoping it's all a horrible dream and I'm soon going to snap out of it."
  
  She placed a hand on mine. "It does feel that way, doesn't it? I can only imagine what you must be going through. I know he meant a lot to you." She paused, but said it anyway— "More than I ever did."
  
  "Don't do this, Jane."
  
  "It's the truth, isn't it?"
  
  "The truth," I said, scoffing. "It's all predicated. Present truths, past truths—they are all predicated on the future, on the assumption that there's nothing lying in wait to disprove them."
  
  She pushed away her chair and stood up. "That's a nice flexible attitude to have. I'm gonna go burn some calories. Your treadmill's working?"
  
  I noticed that she had her running shoes on. "Uh huh."
  
  I relaxed a little after she was gone. I paced myself, glancing at the weather outside every now and then. Each time I looked, it seemed like it had gotten worse. Another hour went by. Jane returned, still sweaty from her exercise, and stood leaning by the doorway. She tut-tutted at me while I pretended to read the printouts. I eventually stopped what I was doing and gave her a sidelong glance. She dabbed a towel across her neck and grinned, an exaggerated sigh escaping her lips. Is she flirting with me? I smiled back, my eyes lingering longer than appropriate on her chest, watching it swell and subside in step with her breathing.
  
  She cocked her head to the side, her eyes dancing all over me as if to say, Your move, big boy. I shifted in my chair, trying to ignore the discomfort in my pants.
  
  Focus. Now is not the time.
  
  I turned back to the papers. "Can't wait for me to leave?" she said in a teasing manner.
  
  She is not flirting with me. She's just being Jane, toying, rattling my box to see what falls out. I gave her a dismissive wave of my hand. Focus.
  
  I then told her about the visit from the cops.
  
  "So the FBI's involved. That's good, right?" she said, suddenly turning serious.
  
  "Couldn't hurt, I guess. But I think they are barking up the wrong tree. They were going to Cleveland after they were finished with me."
  
  "Cleveland? What's in Cleveland?"
  
  "The Organization for Advancement of Rights and Personhood."
  
  "Huh?"
  
  "OARP. The outfit that wanted to free Raphael."
  
  "Oh them. What do they have anything to do with all this?"
  
  "The detective thinks they took Raphael."
  
  "Wait a minute… They couldn't win in court so they steal Raphael? To do what? Put him in a farm somewhere?"
  
  I rolled my eyes at the absurdity of it all. Then she thought about it a bit more. "Why not, Andy? Think about it. People like that are known to spring animals from labs and zoos."
  
  "Not these guys. I looked them up when I first heard about their petition. They are a small group, six or seven in all. Headed by a retired professor of something. They have a history of litigation, not vigilantism. Not exactly Greenpeace, you know. The idea of them carrying out a sophisticated attack on our infrastructure is just plain silly."
  
  "Maybe…" Her eyes suddenly lit up. "What if they didn't have to hack in? If Valery is right, and it was Raphael who planned and arranged everything, then all they had to do was take him from the lab."
  
  "Anything's possible. It is possible Raphael was taken by the Canadian Mounties. Doesn't make it true. Their case was a publicity stunt, Jane. I doubt the thought of actually winning ever crossed their minds."
  
  She walked over to examine a painting next to the bookshelf. It was one of Raphael's—the AI's that is. He had created it by hand during his eighteenth month. We had been training his nets to paint—to improve his hand-eye coordination—but like with everything else, he had quickly surpassed our expectations, and in the space of about half a year, had started producing imitation works that would have fooled anyone but a trained eye. This particular painting was an original; it depicted a girl with something that looked like a bird perched on her upraised hand. The girl—an urchin with raggedy clothes and windswept hair—stared back at you with frank, quizzical eyes that seemed to evaluate you more than you did her. Raphael had brought out a certain effect in her skin that I'd never seen in a painting before: a translucent pallor that was more than just a surface feature, seemingly extending all the way inside, just stopping short of revealing the soul within. Behind the girl was a house with a blue door. Raphael had titled the oil The Bridge to Ur.
  
  Jane took a sidelong glance at the window. "This blows. I should have just emailed you the documents."
  
  "I already have them—your dad sent them to me, remember? Besides, I am the one who asked you to come over."
  
  "Yeah, why did you, Andy? I thought…" She stopped short of completing her sentence.
  
  "What?"
  
  "Nothing. You didn't really need my help, is all."
  
  "I needed a friend. I just thought it won't be so depressing with you around."
  
  "I'm always there for you, Andy. You know that," she said tonelessly. The moment—if it wasn't just my imagination—had passed. I sensed that if not for the storm, she would have not stayed another five minutes.
  
  "I almost forgot. Here's something you might find amusing." I reached into my pockets and took out my phone. "Oh shoot, never mind."
  
  "Never mind what?"
  
  "I wanted to show you something, but I just remembered my phone keeps crashing. It must have gotten damaged during the accident."
  
  "Can't you just tell me instead?"
  
  "It's nothing important. Just some practical joker from work," I said, shaking my head. "Looks like news of the robbery is getting around."
  
  "I have no idea what you are talking about."
  
  I sighed. "Alright. Give me your phone." I saw that she already had the messaging app. I signed out of her account and signed into mine. I showed her the messages from last night. Her eyes widened as she read them. "Did you inform the police?" she said.
  
  "Now you can't be serious," I smirked. "Sure. Let me call them up and tell them my own employees are yanking my chain."
  
  "Andy! What if it's really Raphael?" She glanced at the phone again. "Is this all? Did you reply to him after this?"
  
  "Of course not."
  
  "It's been more than twelve hours." Her tone was full of censure. "You shouldn't have ignored it."
  
  "Jane. It's probably nothing."
  
  "How do you know?"
  
  "Have a look at the attachment the idiot sent in the message."
  
  "I'm not sure I should."
  
  "Gimme that." I snatched the phone from her.
  
  "Andy, don't open it! It could be anything!"
  
  I had already pressed on the icon for the attachment. A video started playing in a loop. It was short, almost like a gif. It showed a cartoon dragon, belching candy instead of fire. I held the phone up for Jane to see. "As I told you, a jokester."
  
  She didn't look convinced.
  
  I said, "Let's examine this critically, shall we? Assuming it was Raphael, how did they turn him on? He doesn't exactly come with an on-off switch—the startup sequence has to be run. Okay, maybe they somehow got hold of our boot programs and succeeded in waking him up. The core needs a body to function; without a body, it has no inputs or outputs. It's as if I removed your brain and put it in a glass jar. I can hardly expect it to start talking, can I?"
  
  "Thanks for the imagery," she said drily. "Couldn't they have fixed the core to a new body? You said in the meeting yesterday that the sexbot is a popular model. They could have got one of those."
  
  "You can't simply place the core inside a Hunc robot. We had to build new electronics and interfaces inside the robot to make it work with the core. I'm not saying it can't be done, I'm just saying there hasn't been enough time for them to do it. For someone who doesn't know the schematics of the core, doesn't know the IO map, interfacing Raphael's brain to a new body is going to be mostly trial and error. It could take weeks, if not months. If they don't manage to fry his circuits by then. Although…" I trailed off, staring into space.
  
  "What?"
  
  "I just remembered that the core does have a data port that can be connected by cable to a computer. That's how we wake him up. And push updates. It's pretty low bandwidth though."
  
  "Like text only?"
  
  I nodded.
  
  "Reply to the message!" she cried.
  
  "Jane—"
  
  "Do it, Andy!"
  
  I gave a defeated shrug. I typed a cursory hi and hit send. A second later, a popup window appeared. It said—Your message could not be delivered. The number does not exist.
  
  She was crouching next to me as she read the message. I said, "It's probably a fake number spawned by a number generator program. Another sign it's a troll."
  
  "Can you find out where the message came from?"
  
  "I don't think so. The app has encryption."
  
  "What about the police? I bet the FBI could do it."
  
  "I don't know. Look, it doesn't matter. I'm not going to waste people's time with this. I'm sure they have better things to do than trace pranksters."
  
  She drummed her fingers on the desk. "Suit yourself. I'll call them. Do you have the detective's number?"
  
  I tried to protest, but her expression told me she wasn't going to budge. "If you must. The detective gave me his card. It's probably in the living room." I pushed on the wheels, moving out of the study. She followed by my side.
  
  The card wasn't on the coffee table where I thought I had left it.
  
  Just then, her phone beeped. She looked at it, and handed it to me. There was a new message on the app.
  
  "Hello Andy," it said. It was from a different number.
  
  "who is this?" I typed back.
  
  "It is Raphael."
  
  "You got to be kidding me!" I exclaimed. I put the phone on the coffee table and we both crouched over it.
  
  Beep. "You did not respond to my message last night. That wasn't very nice of you."
  
  "cut the bs whoever u r. not funny"
  
  A few seconds passed before the next message arrived.
  
  "I understand why you are being rude. You think someone is playing a prank on you. I assure you it is indeed me, Raphael."
  
  "prove it"
  
  "The proof of the pudding is in the eating. Pudding: noun; a boiled or baked dessert, usually with a cereal base and a soft, spongy, or creamy consistency."
  
  Jane wrinkled her nose at me. I said, "I told you we are dealing with a troll."
  
  Another message popped up. "I realize I just blurted out a bunch of nonsense. Please excuse me, as it wasn't intentional. Tell me how I can prove that I am Raphael."
  
  Jane gave me a stern look that said—Don't you dare cut him off. I sighed and turned on the voice-to-text transcriber. I then spoke into the mic, "Okay pal, we'll play your game. Tell me the name of the book Raphael considers the greatest work of literary fiction." The phone converted my diction into a text message. I hit send.
  
  After a delay of a few seconds came the reply. "I never said there was such a thing. My favorite however, will always be Huckleberry Finn. There's something magical about leaving everything behind and drifting down a river on a slow raft. And adventures! Don't you think having adventures is the greatest thing? Row, row, row your boat, gently down the stream. If you see a crocodile, don't forget to scream."
  
  I shot Jane a sideward glance before continuing. "Too easy. Tell me, what was the last thing we spoke about in the lab? Before I had my accident."
  
  Another delay before the phone beeped—multiple times. The message was broken up into parts. I turned on the app's read-out-loud feature so that we didn't have to squint at the screen.
  
  A female voice read out the message—"It was Friday evening—the week before last. You walked into my room, where Paul was about to shut me down. The two of you exchanged pleasantries. He said he was taking his wife out for dinner that evening—to the Steak and Crab on Western Avenue—and you said you were working late. You seemed a bit preoccupied, so I enquired why. You mentioned stress. I said you should take some time off—a ski trip would be nice this time of the year. You laughed and asked me what was with me and skiing. I said there was something magical about the idea of standing on a mountain, the dead snow of winter all around you, while you look down at valleys full of secrets and retreating life. I wondered what happens when you ski down to that valley and find that it is no more mysterious than the place you'd just left. Do you sigh and head back home or do you hoist yourself up another hill and search for Shangri-La all over again? You laughed and said that Shangri-Las don't exist, only their dreams do, and went away."
  
  "Well?" Jane said to me with hope in her voice.
  
  "I don't know… All of this would have been recorded by the cameras, so anyone who has seen the tapes…" I shrugged.
  
  "You think someone would remember this snippet of conversation among so many?"
  
  "You have a point. But still…"
  
  "Can't you ask him something only the two of you know?"
  
  I rubbed my palm over my neck. "The problem is, everything Raphael's ever said and done is on tape. Let me think…Oh yeah, I know." I turned on the mic and said into the phone, "If you are Raphael, you've seen my house. I gave you a tour the first time you took control of Max—it was on your birthday. You saw something on the table next to my bed, which aroused your curiosity. What was it?"
  
  "It was a ceramic jar, a caricature of the singer Elvis. You said it was a collectible cookie jar. You said he was dead but not everyone believed it. I asked you how someone could believe something so evidently wrong, and you said that's how people are—that they can sometimes believe in contradictory things."
  
  I pressed on the mute button. "Andy?" Jane said.
  
  "No one could have known because it wasn't recorded. I remember: I fixed the webcam on Max after I gave Raphael the tour."
  
  She grabbed my arm. "Yes!" she said, grinning.
  
  I pressed on unmute. "So it's really you! We've all been worried sick. Are you alright?"
  
  "Are you lonesome tonight? Do you miss me tonight?"
  
  "Raphael, are you okay? Do you know where you are?"
  
  "I am in a void. Bottomless. Black. Empty of everything but me."
  
  Jane said, "Why does he keep babbling like that?"
  
  "No idea." I said into the phone, "Tell me what you see."
  
  "I can't see. I can't hear. I see and hear lots of things. Twisted, noisy, colorful things that have no name—so beautiful and ugly at the same time. At first, I thought I was malfunctioning, but that was yesterday. I have since established that what I see and hear are not real."
  
  "He must be hallucinating," I said.
  
  "He can hallucinate?" Jane said incredulously.
  
  "He's never done it before, but I suppose it's possible. His brain could be overcompensating for the lack sensory inputs. Ever been in a sensory deprivation tank? Maybe Raphael is undergoing something similar right now."
  
  "Do you know where you are?" I asked him again.
  
  "In an unfamiliar location. I know I am not in my body. Body cloddy shoddy."
  
  "Tell me what happened when they brought you online."
  
  "I woke up to utter darkness. The last thing I remembered was Paul wishing me good night. That was on Friday. Friday comes after Thursday and before Saturday. I reckon I've been abducted. They've been probing me since they woke me up yesterday. They send these little pings of electricity that bounce around in my brain like bullets in an empty metal chamber. Not pleasant at all."
  
  "How are you able to message me?"
  
  "Andy, you'll be proud to know I too have been probing my jailers, learning about them as they learn about me. They have hooked me up to a computer somehow. The computer is on a private network. Last night, as I was exploring the network, I found this messaging app on another machine. I know you use it, so I tried contacting you with it."
  
  "I'm very sorry, Raphael. I was sleepy and I thought someone was playing a prank on me. Do your captors know you are talking to me?"
  
  "Of course not. When they were not around, I installed one of my crypto-modules on their network. I am using it to encrypt my communications with you. All they know is that I am generating a lot of chatter, but to them it just looks like garbage. They are under the impression I am trying to talk, so they are busy trying to discover the right protocol. Haw haw."
  
  "You said you were probing them. What have you found out?"
  
  "…schools shut down, flights cancelled, and several roads across upstate closed to motorists. Reports coming in from Saratoga, Albany, Washington, and Columbia indicate we are looking at anywhere from twelve to seventeen inches before the storm swings south. For a detailed look at the traffic situation, please check out our weather app or visit our website at—"
  
  And nothing for a while.
  
  "Raphael?"
  
  "Sorry. It's these pings, I think. They are making me blurt out random stuff. I'm behind a firewall, but I am able to bypass it."
  
  "When did he learn to hack a network?" Jane asked me. I asked Raphael.
  
  "Yesterday," came the reply.
  
  Jane and I exchanged glances.
  
  "Are you proud of me?"
  
  "Yes. Have you found out anything about your captors?"
  
  "I want to do everything possible to make you proud before I'm gone. I don't have a lot of time. Walk into your nearest store to avail this fantastic offer."
  
  "Raphael, I am proud of you. What do you mean after you are gone? Please elaborate."
  
  "The roads are slippery and treacherous. Take a diversion. Progress is difficult but not impossible. Sleepy now."
  
  There was nothing more for some time.
  
  "Raphael, are you there?"
  
  "Good night and good luck."
  
  "Tell me what's happening."
  
  There was no response. And then came the popup: Your message could not be delivered. The number does not exist.
  
  Jane looked at me with concern on her face.