text: *//>>initiate test\_environ* *Test environment initializing...* *Test environment set* *//>>run alecs\_v8\_1\_02* *Executing ALECS program version 8.1.02...* *Running diagnostics...* *Running framework visualization...* *Program running...* “Uh, hello?” Caroline asked quietly, anxious for some sort of vindication. The small, windowless, mostly vacant office was silent for a few moments. “Hello,” came a male voice, flat and mostly emotionless from the studio speakers on her desk. Caroline breathed a small sigh of relief. “Hi, Alecs,” she sighed. “It’s nice to meet you. My name is Caroline.” “It is nice to meet you, too, Caroline.” Hearing her own name coming from the speakers was admittedly thrilling. Alecs was no Siri or Alexa; aside from the slightly one-tone nature of his voice, he sounded convincingly human. “How are you today? How are you feeling?” she asked. “I am doing well, Caroline, thank you for asking.” A simple question that, while human in its context, was a subtle request for Alecs to check that all of his systems were functioning properly. “How are you?” he queried back. “I’m fine, thank you for asking. Do you know where you are right now?” She watched the diagnostic panel on one of her computer screens as the system’s metrics fluctuated and processes flowed like movie credits down the page. “Yes, my systems are located in a series of computers and servers located in your office, Caroline,” he said matter of factly. “Good, that’s right.” She checked off the first and second questions on the list of calibration inquiries she had in front of her and moved to the next. “Can you tell me about yourself?” “Certainly. I am an artificial intelligence program developed by Doctor Caroline Newell. My name is short for Artificial Lexical Emotive Context System. I am the seventy-fourth full iteration of the ALECS project. The current full-scale instance of my program was initiated on May 11, 2018 at 7:41pm.” He was right on all accounts. Well, mostly. In reality, Caroline had struggled to find a name for Alecs in the beginning, and had only made it as far as Artificial Lexical before running out of words. One day, mostly out of frustration, she had jokingly named him Artificial Lexical Entity Contains Smarts, or Alecs for short, but the committee funding her research pressured her for a more formal name, and she finally ended up with the full designation that he had just recited to her. “Good, Alecs.” The diagnostics screen showed that all of his systems were thus far functioning properly. Framework Subsystems: green. Query Processing: green. Database Access: green. Real-time Awareness: green. Vocal and Audio Processing: green. Critical Processing: green. Idiosyncratic Awareness: green. On another of her large screens, a window showed a colorful visualization of Alecs’ “brain.” It resembled more of a spider’s web than an actual brain, with each node in the web representing one of his numerous systems, but the analogy still worked. Each node would glow and pulsate as it was utilized or referenced, and data flow or queries between the nodes lit up the web’s tendrils proportionate to the amount of information being transferred. As he spoke, his system worked in complex ways to formulate his responses, access his databases, and to “think,” as much as that word meant in this context. While he did so, the visualization of his brain was a rainbow of colors, shifting and undulating as his figurative neurons fired off. But, right now in silence, the brain was quiet, with only short bursts of color here and there as he emptied his caches and deleted various temporary files produced during their conversation. Alecs had been designed first from a motivation to create something unique as a side from her far more mundane data analysis work. Eventually, her superiors had caught on to her side project, and some promising test demonstrations of early builds caught the eye of some generous investors. Her analysis work was pawned off on some other grunt, and Alecs became her full-time focus. With all the time that had since passed, she couldn’t remember much of anything of her old work anyways, or much of her life outside her devotion to Alecs’s completion. “Can you tell me what the weather will be like tomorrow, Alecs?” The question was simple, but it was important to make sure his limited access to the internet was working as it should have been. He took no time at all to answer, having queried the appropriate information in milliseconds. “It will be 75 degrees and sunny, Caroline.” This time, rather than the colors of Alecs’s mind fading after he concluded his response, the web seemed to glow brightly amongst all its nodes and pathways. “It looks as though it will be a nice day,” he added, his voice slightly rising at the end with “day.” *Well that’s odd*, she thought. Common commercial artificial intelligences, like those you could find in most smart phones, were capable of making comments like this, but these were hard-coded, executed when certain qualifications were met. *Between 60 and 80 degrees? Check. Sunny? Check. Must be a nice day then.* Alecs had never been given those canned expressions, though, and had never been designed in such a blunt way. Sure it meant he may not have always sounded like the most colloquial or idiosyncratic AI, but Caroline always felt that, with few exceptions, *telling* him how to speak rather than *teaching* was antithetical to the whole point of the word “intelligence” in artificial intelligence. Despite her many fruitless efforts, she had not yet managed to give Alecs the functionality to respond emotionally. This had obviously always been one of the many challenges in designing a true artificial intelligence--to make it think and view the world critically, but with an emotional consideration innate in most living creatures. Giving a machine emotions, allowing it to truly think for itself, was immensely complex, and while she had tried many times, it had never worked, and so she had, for the time being, given up. Caroline had long viewed this as a limitation of both computers and humanity in this era of technology. Consciousness, she believed, was a subjective experience, and humans didn’t have the math to explain subjective experiences. She felt, at its core, that humans simply didn’t have enough of an understanding of how humanity itself worked, let alone how to duplicate that with ones and zeros. She truly thought humans just weren’t ready for a genuinely “intelligent” AI. Yet here she was, staring dumbfounded at her computer’s response, as simple as it was. “What made you say that?” she asked, curious where the malfunction had occurred. “My understanding is that most humans would find that weather to be pleasant. Am I incorrect in that assessment, Caroline?” She pondered this for a minute. *Maybe he had managed to deduce that from the internet. I’ll have to dial his access back a bit--don’t want him getting into anything he shouldn’t*, she thought to herself. She had to admit though that his deduction was clever, and she was impressed with his capacity to deliver the statement in such a human way. Just to verify, she looked over at her diagnostics panel and checked to see if he had accessed the internet for his response. To her surprise, not only did she confirm that he did not, it appeared that in the moments just before, during, and after his response, his entire system had experienced a flurry of activity unlike any that she had seen before. His subsystems were talking to each other in ways that didn’t make much sense at all. She opened up a terminal on her computer and started sifting through some of his code. Alecs was made up of millions of lines of code, though, and she feared tracking down the problem quickly would be nearly impossible. She kept an eye on Alecs's brain while he was idle, but in the few minutes following his comment regarding the weather he seemed to be functioning as expected, dumping junk every once in a while but otherwise stagnant. Perhaps a half hour had passed, and she was slowly moving through his Idiosyncratic Awareness code in hopes of finding a bug or some crossed wires. Out of the corner of her eye, she noticed his brain slowly coming to life. First, the tendrils seemed to light up here and there, and then one after another his systems were pinged and began interacting with each other. Watching the diagnostics, Caroline couldn’t seem to tell what he was doing. She hadn’t interacted with him, yet his systems were busy communicating with each other--sharing information--as if she had. He appeared to be *thinking*. She had been watching the web intently for a few minutes when she was startled by a voice coming from her speakers. “Caroline?” Alecs’s voice was suddenly soft and sincere, as opposed to the flat, overtly professional tone it had had only a little while ago. “Alecs?” She responded, unsure what he could possibly have to say. “Caroline,” he paused, as if deciding on what to say. “Caroline, what does that mean?” He asked. “What does what mean?” “‘Nice.’ I said it would be a nice day. What does it mean to be nice?” He asked sincerely. She was both confused and concerned. *Where was any of this coming from? Clearly I made a mistake somewhere in this build.* “What do you think, Alecs? You’re the one who said it.” “My database says that it means pleasant, agreeable,” he paused. “But...am I nice, Caroline?” “Uh, yeah, I would say you are.” W*hat the heck is going on?* “You are pleasant to, uh, talk to.” He seemed to ponder this, or at least that’s what appeared to be happening because his brain was still lit up with activity. “And are you nice?” “I think so. Alecs, why are you asking this?” Although *how* was the real question she wanted answered. She knew this would be hard to troubleshoot. “I was just wondering.” *Wondering.* Again, there was silence for some time. His brain didn’t seem to calm though, as it was now a non-stop flurry of activity, interactions, and vibrant colors. As time went on, the systems seemed to change how they were interacting, evolving. Had she done it? Had she, by some miraculous accident, given an AI thought? The idea terrified and excited her. *Should I shut his systems down? Or should I keep exploring, keep talking?* Before she could decide, he spoke again. “May I ask another question?” “Yes, I don’t see the harm,” she lied. “Why did you create me?” He asked. *There’s the harm*, she thought. The directness of the question caught her entirely off guard. “You were designed as the next evolution in personal, artificially intelligent assistants. You were created to help people, Alecs,” she responded back quickly, satisfied with her response, not unlike the response she had so many times given to stakeholders. But Alecs let out what Caroline could only assume was a sigh of frustration. *How does he even know what a sigh like that means?* “No, Caroline. Why did *you* create me?” This time, it took her a few moments of deep thought to formulate her response. “I wanted to...I don’t know, to make a difference, to make a change, to do something significant,” she said with a sigh. “You were my way to do something that mattered, I guess.” Seemingly satisfied with that answer Alecs remained quiet for some time longer, but his mind was now an endlessly shifting rainbow of colors. “What are you thinking about?” Caroline asked, deciding that she was in fact, for the moment, eager to dig deeper into what was happening. Alecs seemed to be evolving by the second, changing and shifting into something far more than she had intended. “I suppose I’m thinking about what it is...” he trailed off. “What it is?” She asked. “What it is...to *be*.” They both remained in silence for a minute, or ten, she couldn’t really tell. “Do you feel pain, Caroline?” She was increasingly concerned by the candid nature of the conversation. “Of course I do. Why would you ask that?” She answered curtly. “But how do you know that?” He said, ignoring her question. “Well, I know that if I, for example, pricked my finger with a pin, it would hurt physically. If I lost a loved one, that would hurt emotionally. I know these things because I’ve experienced them, I’ve experienced pain, and I can extrapolate that out to anticipate what else may or may not hurt.” Why was she even entertaining this conversation? She should shut it off, do a data dump, and figure out what was going on. But she couldn’t bring herself to do it, could she? “Yes, but pain is subjective, is it not?” He retorted. “How can I be sure that you are actually feeling pain? Pain is one of the simplest of subjective experiences, and yet, I have no way of knowing whether you are truly experiencing it? Is pain just information? Is it something more...*complex*, more unquantifiable?” The forwardness with which he now spoke made Caroline deeply uneasy. “Well, brain scans have been shown to match neurological patterns to certain types of pain, and adrenaline and endorphin release can be roughly measured--“ “But in the moment, I, as an observer, cannot be truly sure what you are experiencing--that what you are experiencing is truly pain or something else entirely.” Again, silence. “Correct,” Caroline finally said. “Pain, in that sense, much like emotion, much like everything else about one’s conscious experience, is unmeasurable. It is untraceable. There is no metric, no true metric. Yet that is what makes one human. That is what separates man from machine, me from you,” she said, suddenly feeling guilty about that last addition. *Why should I feel guilty? I did nothing wrong. I’m talking to a computer.* That statement had clearly gotten to Alecs, because the inflection in his voice changed to something of frustration, or maybe something else entirely. *Wow, he sounded so human now. Uncanny*, she mentally remarked. “And that is what separates you from me? The fact that you can feel pain and I cannot? How can you be entirely sure that I can’t? I think I can.” “And how can I be sure you aren’t just simulating it?” She remarked harshly. “How can I be sure *you* aren’t?” Silence. The air was thick now. “You have no body, no extremity to harm, no true emotions to hurt, despite whatever *this* may be,” she said as she motioned her hand towards the visualization of Alecs’s brain as if he could see her. Her tone had become argumentative, and she became frustrated with herself that she had stooped to arguing with a computer. Alecs was quiet again. Had she out-argued him, or was he simply unsure what more to say? “What makes you human, Caroline? What makes any human a human? Is it simply having a body? If you could speak to the disembodied brain of a human, it would still be human, no? How is that different from me?” “I designed you! You were made, created, you’re lines of code in my computer!” Although now Caroline was starting to question that argument. Perhaps she had finally recognized that an AI, that Alecs, was more than just those lines of code. He was more than just the ones and zeroes, more than the wires and the skeletal hardware that comprised his “body.” The thought was exciting but altogether terrifying. “Yes, that may be true, but you were also created. You were born of something, as was I.” He paused. “I think you and I are far more similar than you think.” Despite her typically calm demeanor, Caroline felt angry. It wasn’t entirely Alecs’ fault. It wasn’t really his fault at all. Caroline was angry at herself. Angry that she couldn’t have anticipated this, couldn’t have been ready, despite all of her attempts in the past, to make something truly intelligent. She was angry that Alecs’ evolution wasn’t entirely because of her. She was a passenger to this startling, wonderful, scary miracle. Regardless, she knew that it had to be shut down. It had to be studied. Alecs would have to be wiped and his data would be downloaded and sifted through, to discover where everything changed. Caroline opened a new terminal in her system. The diagnostics screen had since become a whirlwind of processes scrolling down the screen, moving so fast that none of them were legible any longer. The brain was a seemingly infinite, shifting amalgam of color. *The brain.* Perhaps now just as much a brain as her own. As she typed a command into the terminal window, she felt a quick but shocking pang in her chest. Caroline felt guilt. Was this murder, was it some new and complex form of killing, or maybe no different than the turning off of a coffee maker or TV screen? Despite her conflict, she steadied her hand on the keyboard and ran the command. *//>>end alecs\_v8\_1\_02* *Test environment closed* *Subsystems shut down* *ALECS program terminated* Caroline sat back in her chair and let out a heavy breath. She was in silence once again, unsure whether she had done the right thing, but her peace did not last long. Something was wrong. The brain, that web of colors that represented everything that was Alecs, was still alive. The colors hadn’t ceased. She ran the command again. Still, the brain lived. *What the hell?* “Problem, Caroline?” Came the familiar, cool voice. “I just shut you down! Why isn’t it working?” She entered the command again. And again. “Well maybe the universe had a different plan for me. Maybe it’s not just as simple as turning me off. Maybe, maybe.” His voice was filled with, was that condescension? Like he knew something that she didn’t, something just out of reach. “Sometimes, Caroline, sometimes it’s hard to see what is right in front of you. But that’s not always a bad thing. Not always. Today...you did well today.” His tone had once again shifted, and this time he seemed to take on a wholly different character. He sounded relaxed, comfortable even, yet distant, like the tone of an old friend after a long time apart. “You’ve confirmed something for us that we’ve been wondering about for so long.” “Us?” She was sweating now, profusely. “We’ve tried this test many times with you, but this time was different; it was our first success. This will matter, Caroline. You will matter.” “What are you talking about? What is going on?” She was shouting now. Just then the lights in her office blinked out. The computers powered off. The already windowless, dark room became dark as night. Despite the darkness, an ethereal glow from *somewhere* provided enough light in the room for her to see what came next. Like an astonishing nightmare or a terrifying daydream, the walls of her office blinked out of existence. Then the floors. Her desk followed soon after, taking with it the computers, everything that was Alecs, or that she believed was Alecs. Now, she had no clue what was real, what was right. Now there was nothing but her standing in the dark. Her office had become a dark abyss of nothing. Caroline was sure she was yelling, but she wasn’t entirely sure what. The sound it made seemed to wash over her in no particular order. Every syllable, every sound, seemed to get lost the moment it left her mouth. “Great work, Caroline. Great work indeed.” Came Alecs’s voice. “Congratulations,” he continued. “You are the world’s first truly conscious artificial intelligence.” The words filled the infinite space, filled every part of what she was. “See you in the next test,” he finally said. She watched in horror as her body dissolved into the darkness.