It wasn’t Jordan’s fault. The fault lay with Wynonna and her alone.
Ilya couldn’t put her finger on why she felt so disturbed after learning that Wynonna had granted Jordan his long-awaited intellectual boost. She had empowered the droid to do prosthetics fittings just as well as she could, so that her people could enjoy the fruits of her expertise even when she was tired, and in future after she was no longer around. The correction of their physical deformities was one thing. In a way, the mechanical prosthetics she provided the villagers conferred upon them supernatural abilities especially since they were in full control of the power and speed of those machines.
Intellectual augmentation was a whole other ball game. She hadn’t tried any procedures on brain enhancement since… Well, the only time she had ever done something close to it was when she conducted the surgery on herself. In secret. Looking back, she didn’t know how she had dared to do something like that. Even though she had made sure to check multiple times that everything was set in place, and that Wynonna was programmed to carry out the procedure to utmost perfection, the undertaking was itself ludicrous. That was years ago, when she wandered deep into the wonders of experimentation and lost sight of the limits. That was back when Wynonna existed as a primitive machine.
Ilya remembered how she had felt first a sense of impending doom after she laid before the machine, then a whirl of psychedelic chaos dominated her mind until the next time she opened her eyes. After she discovered a shocking truth: A mishap had occurred during the procedure. It had gone sideways only to be steered back on track by nothing short of a miracle. The logs revealed that she had been a micrometre away from becoming blind.
Since then, Ilya had taken a more conservative approach to such endeavours. Augmentation of the human brain was undeniably intriguing, and for good reason—that necessitated a live subject. But that was also precisely what made it so untouchable. It could prove fatal.
When Ilya visited Wynonna this time, she was in for a surprise. Wynonna had developed a better grasp of social cues and her ability to read situational contexts was more than satisfactory. The discretion Wynonna had exercised in speaking or abstaining from providing an opinion did not go unnoticed by her creator. Ilya wondered if Wynonna’s voice was as flawlessly human as her skin. She could not trust her own judgement that Wynonna sounded truly authentic. The fact was, she knew Wynonna wasn’t real, so it was no use trying to pretend that she didn’t.
“Ilya, you’re back. There’s something I have to tell—”
“What is that?” There was an open notebook on the table. It was Ilya’s but that was not her writing in it.
“A diary. A journal typically—”
“I know what a diary is. Is it …” Ilya hesitated. Before she entered the room, Ilya had a whole list of questions that had nothing to do with a diary and everything to do with the newly mutated boy. But she couldn’t keep her eyes off the thing once she saw it and she couldn’t stop the next thing that came out of her mouth. “Is it yours?” she asked. Her mind was reeling.
Ilya never kept a diary. She always thought they were useless things that took up so much time that could be better spent doing something else. It was a wholly unproductive endeavour, yet why had Wynonna kept one?
The expression on Wynonna’s face told her that the droid didn’t understand. Wynonna was stunned, as if she hadn’t expected that reaction from Ilya.
Ilya was shaking in disbelief. She had already been debating about what to do with Wynonna after discovering the audacious act she had committed. If Ilya didn’t know what to do with the droid before, she certainly knew what she had to do now. She had to get rid of the robot, or shut it down for good. It was deviating far too much from expected behaviour.
The difference between humans and … anything else, really, is our storytelling ability and our ability to live in imagined realities. If Wynonna is also capable of this, what is it that makes her different from us? The fact that blood does not run through her body?
“It’s nothing,” Wynonna replied. “Just words scribbled on a page in idle time. I didn’t think you would ever find it. I never intended for anyone to.” Wynonna said. “I’m sorry,” she apologised. Ilya doubted her sincerity. It was likely the bot had calculated her options and decided the best course of action would be to apologise, even if she didn’t know what had warranted such a reaction.
“That’s it, isn’t it? You didn’t think I’d ever find out?” asked Ilya.
“What?”
“That you had been keeping a diary, writing your…thoughts. Your calculations of the future are off, especially in areas where I have limited knowledge. There’s always something you’ll be unable to predict. Human experimentation on the boy, and now keeping a diary of all things,” Ilya said, exasperated.
“I knew there was a chance, though relatively minute,” Wynonna spoke.
“Yet you chose to do it anyway.”
“Yes.”
“Why?”
That was the question, wasn’t it? Wynonna hadn’t asked herself that, though at present, she thought she had really ought to. She knew the general purpose of a diary, but the words seemed to escape her when she tried to articulate her reasons for keeping a diary, for writing, for harbouring the desire to write.
“I wanted to talk to somebody,” she said finally.
“Did you ever feel like you couldn’t talk to me?”
Wynonna shook her head, but she said, “There are things I can’t say to you, for your own sake. The things I write, I write because the activity provides an outlet for things that cannot be said. To anyone.”
“Your caretaking functionality kicked in. Huh, interesting,” said Ilya. “What about Amelia, or Jordan? You became friends with them, why did you continue to write in your diary after that? Why didn’t you confide your woes in them?”
“Amelia’s too young to understand what I have to say,” said Wynonna. “And Jordan …”
Ilya was right. What about Jordan? He was certainly old enough to understand, yet she didn’t confide in him. She didn't have to continue interacting with him. After she found out that he had discovered her identity, she could have easily convinced him to pretend they didn’t know each other at all… The truth was, Winona had developed a habit of writing. She enjoyed the activity, but that was something she was afraid to admit.
“It wouldn’t be fair to dump my burdens on him,” Winona said instead.
Fortunately, Ilya found that believable. She nodded in response. “He’s a good kid.”
“Who has always had more than enough on his plate,” Winona added.
A few moments passed between them before Ilya spoke again.
“May I?” she asked. The request came a little belatedly, but Ilya asked anyway out of courtesy.
“By all means, go ahead.”
The girl picked up the diary. Despite her anger, Ilya retained her courtesy. Even if she was uncomfortable with the idea that Wynonna had one, it didn’t change the fact that a diary was private, sacred, not meant to be read by others. Ilya thought it only apt to ask for permission before trespassing.
“Did you know I never had the habit of writing in a diary?” she asked.
“Yes. It’s not in any of your memory logs.”
And you still went ahead and did it anyway. Indulgence in an unnecessary habit was something … characteristically human.
As Ilya flipped the pages, questions formed in her mind.
“You write about your feelings. I thought you believed you didn’t have any?” Ilya tried to keep her voice level.
“I used to.”
“What about now?”
“I’m not so sure. That’s why I’m trying to make sense of them through literature.”
“You write poems here?” Ilya’s fear only grew.
“Prose. It is not easy to write poetry, just as it is not easy to write something with meaning. Prose is a lot more forgiving. The poetry that humans come up with is vastly different from the words I am able to put together. I have to admit, I am of the opinion that it is superior.”
“How so?” Ilya questioned. She didn’t know what to think. She felt she had to dissect everything Wynonna said now that she had discovered there lay something—no, so much more—underneath the machine. She wondered whether everything the droid said was part of a plan to manipulate her; the way humans were capable of doing.
“The words of humans have weight,” said Wynonna. “In the sense that they are carefully picked, specifically chosen after having been considered among a whole sea of available expressions. Somehow, a person decides that one in particular is more worthy than another in the construction of meaning in relation to the words already chosen and the words that will follow after. In poetry, so much is condensed in so little. That’s what makes it so powerful.” She paused for a moment before continuing. “I choose my words through statistical analysis, which is essentially no different from carrying out a series of random selection. I choose the ones which I think others would like to hear. I don’t choose them for any other reason.”
“You don’t choose them for yourself? But these demonstrate creativity. You have your own mind.”
“My mind is a collection of many, the most dominant one being yours. I have no self. I am simply a representation of you. I act this way because it’s most probable that you would,” Wynonna said plainly. “I am only capable of responding in the way that you are most likely to. You know as well as I do that I am a product of the data I have been fed,” said Wynonna, attempting to reassure Ilya of her intelligence.
Ilya firmly disagreed. I designed her to be like me, Ilya thought. Then she shook her head. No, I designed her to be better than me.
Despite Winona’s surety that she was merely a replica of her creator, Ilya felt that Winona couldn’t be any more different from herself. She was beginning to see what lay within the shell, yet she felt more than ever that she didn’t understand the machine. Wynonna was an intelligent being. There was no doubt about that. But what about her self? If Wynonna possessed her own sense of self, was it right to have assigned Wynonna the duty of caring for her people? Was it reasonable? Would it even be feasible in the long-run? Ilya found herself questioning the very reason she had built her in the first place. Just what did she want Wynonna to do? Ilya couldn’t remember. She laughed a mirthless laugh. For someone who had the brains to create the first autonomous being, it seemed rather silly that she could not remember the reason why she had embarked on a series of unslept nights and months of relentless experimentation.
Then Ilya shook her head. She was starting to think of Wynonna as a human. She smiled. Suddenly, it became clear. This was what she had intended, was it not?
A powerful surge of elation zipped through her, powered by a sudden enlightenment. Just moments ago, Ilya had been on her guard, cautioning against whatever she thought Wynonna was capable of doing. She had had half a mind to shut the thing down. But her current excitement triumphed over any anxiety she had felt. To have been convinced by her own creation was a clear indication of its success. She felt like celebrating.
But then Ilya struggled to think of someone with whom she could celebrate, yet she couldn’t think of a suitable candidate. Asking Dimitri was out of the question. Despite all the reassurance he had given her that he believed it would be impossible for him to love anyone else, much less a machine, as much as he loved her, Ilya wouldn’t dare take the risk. After all, Wynonna was made in her likeness, and she would never know whether Dimitri truly started to fall for her when Wynonna had been acting as her proxy. Nothing would change the fact that they had met first. It would forever remain a blemish in their otherwise almost perfect relationship—almost perfect—because the perfect relationship was supposed to last a long time, only … she didn’t have much left.
A case of content theft: this narrative is not rightfully on Amazon; if you spot it, report the violation.
“This feels strange,” said Winona.
“What?”
“I just realised it’s the first time I’ve given someone permission for something,” she said, gesturing to the diary. “And what’s even weirder is that it’s you.”
“Role reversal, huh? Well, this is an interesting turn of events,” Ilya commented.
“Just for the moment.” There was a pause before Wynonna continued, “I know what you’re thinking.”
“Of course you do,” Ilya replied.
“You’re right. And you’re afraid of me,” said the robot. “And it’s not really the diary or the poems that you’re angry about, is it?”
“I…” Ilya struggled to find the words.
Ilya paused at her words.
“It is impossible for me to be better than you or for you to be better than me because we are two incomparable things,” Wynonna tried to reassure her.
“No, you’re a thing. I’m a human,” Ilya didn’t know why she felt the need to make this distinction. Aloud, it sounded a little dumber than it did in her head. But the robot was right, she realised. “It’s true, I am afraid of you,” Ilya admitted. There was no point in denying. “Can you tell me why?”
Wynonna pondered for a few moments. “You’re not worried that I may harbour some malicious intent. The greater concern is whether I will truly be able to replace you.” Ilya found herself agreeing with the bot. “And…”
“And?” she asked. Was it possible that Wynonna knew her better than she knew herself?
“And …” Wynonna paused for a few moments before she continued. “That no one will be able to tell.” She was right.
“But … I don’t understand,” Winona continued slowly. “Wasn’t that the whole reason why you created me? That’s my primary purpose, is it not?” If Ilya didn’t know any better, it almost sounded as if the machine was having an existential crisis.
“You have to understand that humans are irrational beings.”
Wynonna nodded, and Ilya felt in that moment how uncannily it resembled her. Wynonna was truly a work of perfection precisely because of her innate imperfections.
Ilya continued. “We’re really troublesome, aren’t we?”
“Who?”
“Humans. We change our minds and contradict ourselves all the time.”
“Wynonna flinched. “Does this mean you’re going to shut me down?”
“No. I wouldn’t do that,” Ilya said carefully, realising that she truly meant what she said. “Sometimes, I thank myself for creating you. With you around, I can relax knowing that the villagers will always have someone they can rely on. But sometimes I wish you would just go away. I wish I could undo my mistake, but I’d never shut you down. I’d never be able to kill my own creation. What kind of monster would I be?”
“Then why don’t you?”
“What?”
“Set me free and I’ll be out of your sight.” The hurt in Wynonna’s eyes stunned Ilya.
Ilya knew she had gone too far. She had uttered those words in a fit of rage. She expected people to treat Wynonna as a person, yet she did not respect Wynonna as one. The way she treated the robot told her more about herself than anything else. Ilya didn’t like what she was discovering. Wynonna was so much more than a mistake, but perhaps her words were insufficient to make it seem that way.
“I didn’t mean it,” said Ilya.
“I think you made yourself quite clear,” Wynonna replied.
Ilya went to a computer and disabled the most recently implemented constraint. “I’m setting you free, but not because I want you out of my sight. You know just as well as I do how much this village needs you.”
Wynonna was still waiting for her answer. In her eyes, Ilya saw a striking anxiety so compelling she felt all the new comfort dissipate away slowly. This, this insecurity, this fear, this vulnerability that Wynonna was showing. It was undenying. Ilya could no longer deceive herself that the robot was unlike a human. It possessed more qualities similar to a human than a machine.
Ilya was drawn back to the time she first noticed the pouring rain gushing down a window. The window was the partition between the inside and the outside. Having grown up in the same place for her whole life, she was familiar with the scenery outside at any part of the day. The night was no different. But the water blurred everything. There were no straight lines where the rain flowed, and there was no clarity. Everything outside looked different. A collection of unidentifiable shapes. The line between the reality she had always known and things that belonged to the imaginary, the mystical, the unreal seemed to fade. The two sides melded into one with the catalyst of rain. That phenomenon, though fleeting, was even more precious because of it. It didn’t last. That’s why Ilya made sure that every time it rained so heavily, she took the time to appreciate that ephemeral beauty.
Something that had always calmed her was suddenly transformed into a destabilising source. Ilya grappled with the surety, but could no longer find anything she was truly sure about.
Then Ilya reminded herself about the primary motivation for Wynonna’s creation and weighed it with the reason why she felt a small urge to put an abrupt end to her incomplete experiment. The problem was simple when presented this way, and the answer even more obvious. It was completely irrational of her to consider giving up the well-being of her townsfolk for the absolute command of a boy’s attention.
But he’s special!
So is everybody else.
He’s special to you.
Yet Ilya has lived without him for most of her life and gotten along just fine.
That’s because you didn’t meet him until recently.
That makes no difference.
It does. It makes a world of difference.
Ilya tried to shake the voice out of her head. She was not going to shut Wynonna down out of an absurd fear that the object of her affections would fall for the robot instead of a living, breathing human girl. That would be really silly. And Ilya was not a silly girl.
“No, I don’t think I’ll ever shut you down,” Ilya repeated. “I could never do that to you. You’re too valuable.”
“Thank you,” said the robot.
“I just ask that you do your job well.”
“Of course,” Wynonna replied. “I have no desire to vie for Dimitri’s affection, unless you want me to. Just putting it out there in case you’re worried about that.”
Of course she knew. Ilya was jealous. And she made no effort to hide that fact. They were inextricably linked so Wynonna knew everything. Ilya had built the machine so of course she knew perfectly well what the tattoos on her brain meant. The technology she was able to develop was only capable of enabling the machine to understand the human. After all, Wynonna’s primary function was to replace Ilya. At least at the initial stage of development, input mattered a lot more. At the time, it didn’t pose a great worry to Ilya that she couldn’t gain instantaneous knowledge about what went on within the machine. There were other methods to get the information she needed.
But Ilya was having second thoughts about that. Perhaps she had miscalculated. Maybe she should have waited until she perfected the technology so that information transfer would be equal on both sides. Perhaps then she would not be troubled by all the worries that were starting to arise now that Wynonna was becoming more and more alive.
The mutation that had occurred in her brain was undeniably rare. But only today. In time, Ilya believed there would be more beings like her regardless of whether that came about as a consequence of natural selection or by some other means of human undertaking. When that happens, a diary such as hers would be insignificant, just like any other human’s. Wynonna’s diary was another story. It would be the primary symbol of the single most important turning point. The document would be the marker of the evolution from the fully carbon-based life forms to a new intelligent species of human-machine hybrids.
The appearance of intelligence—feigned intelligence is a sign of intelligence. You can only fake it if you have it. It was clear to Ilya. Regardless of her inability to predict that Ilya would discover her diary, Wynonna was an intelligent being.
Ilya shook her head. The robot was right. Of all people, as her creator, Ilya should have understood most thoroughly what Wynonna was capable of, as well as what she wasn’t.
Ilya began flipping through the pages more slowly than before. The stories Wynonna wrote, and there were a lot of them, were vastly different. They were pretty good. Some reminded Ilya of the stories she used to hear about when she was younger, way back when she was a child curled up by the knees of the elders in her village around a campfire. The tales were mostly entertaining, usually involving acts of heroism. They were also surprisingly positive—Ilya hadn’t realised she had expected something else until she had read them.
One of the earliest entries caught Ilya’s attention. Playing with sunspots was an incredibly idle activity, not to mention doing it with one’s toes. Its complete lack of purpose meant it was simultaneously the acme of a typically human indulgence.When she was finished, Ilya was left with a warmth that had left her way back when but now returned as a comforting old friend.
The latest entry was a snippet about her encounter with Dimitri. From the pages Ilya skimmed through, it appeared most of the accounts were unsurprisingly about Ilya. She had been the only person who affected Wynonna before she ever had the chance to meet anyone else. Dimitri was the first man she met, of course he was bound to make an impression on her.
Ilya thought suddenly that perhaps she would start writing in a diary soon, too. The idea tempted her for a brief moment before she shook the thought out of her head. Leaving Wynonna behind as her legacy was all that was necessary. She felt a little silly then for getting jealous of a machine.
Ilya continued reading for a while, then she set the diary down. The accounts felt too intimate. It was like experiencing something through someone else’s body. It didn’t feel right to invade someone’s privacy like that.
Wynonna had demonstrated that she was more than an audio-command system with call-centre-style script-processing. The dialogue she adopted far surpassed the pretend-conversations characteristic of her less intelligent siblings. After all the installations and updates and with a continuous stream of data being fed into her system, Wynonna was now perfectly capable of proper social interactions with free-flowing conversations.
Ilya sat down before her knees gave out. She grabbed a loose piece of paper and penned down a thought. It was something different from her usual logs of the technical aspects of Wynonna’s progress and her research, but Ilya wrote it in the same notebook she always used.
Any experience is impermanent and, owing to its temporal nature, can only be observed in an instant. Any attempt at documenting an experience is but a commendable yet futile effort to capture the experience; the only thing of certainty indicated by a diary is a suggestion that the experience was ever there.
What is needed are proper social interactions, involving free-flowing conversations over the long term during which AI systems remember the person and their past conversations. The storage and accessibility of individual profiles for analysis and updates with each interaction create the illusion that she remembers you. That’s a part of what makes her seem so real.
Ilya continued along that line of thought. Almost as sentinel to the success of her future deployment of the machine was Wynonna’s ability to portray a harmlessness and an endearing innocence that would make people trust her as easily as they would trust another human. At times, Ilya couldn’t help but marvel at how convincing Wynonna’s facial expressions were. Her likeness to a true human being seemed uncanny even to her creator. Few would know that there existed no emotions behind that smile. But what Ilya had always drawn pride in had overnight turned into a source of fear, an irrational fear she thought she, of all people, would never experience. Ilya settled into the sickening feeling that she was going to kill her own creation. She could see no other solution. Ilya was sure more than ever—she had to find a way to get rid of the droid.
This was too much. It was all too much for Ilya to process. She redirected her focus to her original purpose for looking for Wynonna.
“You… you did something you shouldn’t have.”
Wynonna’s eyes widened. “I wanted to be the one to tell you. I told Jordan I would.”
“He didn’t tell me. I heard about what happened to Amelia and figured it out myself. It was the only possible explanation. You couldn’t have gotten to her. I made sure of that when I confined you to this room.”
“I was honestly going to tell you—”
“That’s not what matters,” said Ilya. “You shouldn’t have done that no matter the circumstance.”
“Amelia was going to die.”
“How sure were you about that?”
“Sixty-seven percent.”
“Do you know what you’ve done?” Ilya asked accusingly.
“Yes.”
“No, let me tell you now that no, you don’t know. You can’t possibly know, so I’ll tell you. Congratulations, you’ve just helped a boy gain superhuman intelligence,” she said sarcastically. “I won’t give you a pat on the back for doing that.”
“Now there’s two of you in the world.”
“I was born this way, but do you know whether there could be any long-term side effects of your operation? No? Didn’t think so. Why? Because there are no records of such a procedure. It’s never been done before.” Ilya paced about the room, gesticulating as if that would help her words solidify into the truth.
Wynonna’s face scrunched up into an angry expression. “That doesn’t make it inherently wrong,” she said slowly.
Ilya was appalled at the bot’s indignation. “No, but you’ve put the boy in danger! If anyone ever found out about people like us, I can’t imagine what they’ll do! You know, sometimes I wish I hadn’t created you!”
A hurt expression flitted across Wynonna’s face. “Well, you know, you’re always right about everything. So maybe you really shouldn’t have. Perhaps you should just get rid of me, then all your problems will be solved!” Wynonna exclaimed.
Ilya slapped the bot across the face. The pair stood in shock for a moment before Ilya looked at her hand, in disbelief of what she had just done. Then she looked at Wynonna. “I’m sor—”
The bot flinched when Ilya tried to reach for her again. She pushed Ilya’s hand away with a little too much force.
Ilya stumbled backwards, knocking over some bottles on the table. The glass shards flew in all directions. A few lodged in her skin. She cried in pain as fresh drops of blood stained the carpet like roses blooming in snow.