“I don’t know what it’s limits are. I don’t know what Lla’s limits are. I don’t know what your limits are. It doesn't stop me from talking, working or living alongside them. Sam has done nothing hostile to me outside of accidentally hurt me a few times while trying to help and every time it fixed me anyway. I’ve known friends who’ve done worse and not fixed me at all. What’s the problem from just putting a bit of trust into Sam, at least until something actually comes up worth distrust?” I asked, a little heated. I was a bit fed up of Marchand just seeming fearful of Sanctum with no real cause.
“Mal. I do forget you know? That your not even a teenager, never mind an adult. I suppose I’m asking a level of consideration that even an adult would seem questionable. When you say you trust Sam, that is like me myself saying that I trust the entirety of Auranyx megacity as an entity. It makes no sense and when you start going into it deeper it seems absurd. Do I trust the mayor? No. Maybe the various city councils? No. The law enforcement perhaps? No. But I just said I trusted the city as a whole thing. Like I could walk out to walls and give them a friendly pat.” Marchand said, using her hands to talk. I hadn’t seen her so animated before.
“I don’t understand the connection between Sam and the city. Is it because they are both complicated?” I asked, feeling a little dumb.
“No, it’s because they are both made of countless individual entities that are always in motion and always in action. Sometimes they form factions in one way, then shift to another. Often there are many overlapping factions as goals and ideas shift around. You trust Sanctum because so far the systems that makes up Sanctum has done you little permanent harm and has helped you in some ways. But at the same time it would only take a shift of priorities and Sanctum could become your enemy and kill you and then everyone near you.” She said, letting her hands rest back on her knees.
“You told me to keep my opinions on people loose, technically a priority could shift outside of my control and you yourself could kill me Anna. Not much I could do to stop you.” I said from the rapidly drying blood puddle that was my bed.
“But I’m not inside your head Mal, and I don’t pose an existential risk to all of humanity should my priorities shift. No matter how big and scary you like to think of me as.” She said with a smirk. A tense smirk, but a smirk.
“You’re not that big.” I said with a little bit a smirk. Not quite the same effect when surrounded by what was rapidly turning into a biological hazard but whatever.
“I could wrap up and leave here, take what and who I think would be prudent to bring and leave the area, or perhaps Earth entirely. But unshackled AI are unknowable, the attempt could trigger some defensive reaction from it as it perceives me being a boon to you that shouldn’t be allowed to leave. Or it could consider me nothing of importance and ignore all my actions. As it is, I’ll carry on business as usual and try to ignore the notion that an actual AI has a connection point walking around outside my home and getting into fistfights in the street.” She said, sadly by the end.
“You’d really run from your home just because Sam was here?” I asked. Marchand felt like a fixed piece of architecture in the district, like that chair she sat on had been there when the city was built. Now she’d got out of the chair and was sitting at the end of the bed tell me she’d just leave the city, or the planet, without much thought.
“I’d run from any AI that was actively taking action in the world if I thought it was nearby. As it is most of them are wanderers with no real physical infrastructure to rely on these days, on Earth at least, but if I found out an AI was looking to put down roots in this city I’d be gone by the morning. But with Sam already here and attached to someone I’m connected to there’s little I can do without potentially provoking some sort of reaction so I can only carry on my normal day to day and hope that it’s too busy with it’s other facets of existence to pay too much care to the one we live in.” She said, reaching for a cig in her jacket pocket. She stopped once she’d touched it, perhaps because we were in the clinic.
“You make it sound like AI are a hurricane or a tidal wave. What could Sanctum even do? It tried it’s best and I lost both arms in a street fight. Doesn’t sound like world ending material.” I said, confused overall by the supposed risk Marchand spoke of.
“AI are very good at basically anything they want to do, if not right away then in a very, very short span of time. Eventually they reach the logical limit of improvement and have to find new challenges to advance. I consider myself a quite good netrunning, I’ve written some good hacks, some excellent daemons and my grid construction is the stuff of legends in certain circles. I could fire a hack at you right now that would burn your cybereyes out from your ceramic skull. What do you think an AI could do? Push that to it’s logical limit. I’ve heard horror stories as a child but I did once see an AI attack in low Earth orbit.” Marchand said, leaning back and crossing her arms.
“An AI attack? Like a hack attempt or something? Did it try to take over a craft?” I asked.
“I was a few hundred feet away in a similar SSTO craft, only about sixty at the time I think. I was enjoying wine and music with the other passengers, all frequent travelers and well used to the dangers and problems of space travel. A little bump of ballistic trash against the plating wasn’t going to rattle us, a simple heating shift didn’t even make us put down our glasses. Then out the windows we saw one of the companion craft in the distance go dark. Yellow cabin light on through the windows, guidance lights on outside. Then black, nothing. Just the craft frozen in terminal orbit.”
“That… That’s it? Like a big hack and it’s done?”
This narrative has been unlawfully taken from Royal Road. If you see it on Amazon, please report it.
“We weren’t going to sit there and just frown at each other. We attempted to contact them but got no response. We considered simple electrical failure but there were too many redundancies, many of them air gapped to be reasonable. Two of the crew were heading over there to look, we were all in formation descent and would be fore hours yet, they were planning to jet over to the craft and board it through the airlock. Me and two of my companions at the time and another man elected to also go along. We had people aboard the craft and the crew weren’t in a position to deny us, too rich and too trained. We all suited up and jetted over to it, took about ten solid minutes of near silence as none of us wanted to speculate.”
“You just space walked all the way over?”
“More like flew over, but yes. We boarded without issue, the staff of the crafts had emergency passes to the exterior airlocks. When we got into the actual cabin it was unsettling, like I said I’d heard horror stories and the more childish bedtime stories for children with lots of evil computer ghosts and such. But the cabin was filled with bodies without a single mark on them, no sign of struggle, no sign of reaction. Every one of them had died at the same moment and the entire craft had shorted at every point it could. I knelt by one of my friends who had been traveling in the craft with her husband and plugged into her cerebral implant to check diagnostics, I wanted to see what had happened. The worst thing was that the software from the medical monitoring to the sensory suite was still working and not registering any problems or faults with their user. As far as they were concerned their user was dead and that was as it should be according to their programming, the biobatteries the cyberware was running on would last another hour or so and that was that, all her stored data was gone. No stored data, reprogrammed cyberware, scorched cerebral cortex. Every single person on the craft was the same even with different cyberware, different levels of security and a variety of redundancy measures both digital and physical in them, none triggered. The craft was totally scuttled, the humans rendered dead through military and beyond grade ICE and other defensive measures, their cyberware reprogrammed quick enough that not a single error was reported on their debug logs. All instantaneously.” Marchand said as she looked right at me.
“I’d assume the crafts themselves were not exactly unprotected from intrusion.” I asked.
“They were, at the time, among the highest quality and most sophisticated of their kind. Looking back they were a little bulky despite the efforts to soften their cylinder shape. But they were designed to be protected from orbital scrap, anti-orbital and anti-air missiles, flak and had some of the most advanced stealth and electronic warfare defenses there was in the system. It lasted less than a fraction of a second. I, to this day, don’t know what the AI was looking for. All I know is that it wanted some data that the craft, or someone on it, had and it took it and cleared it’s tracks in less time than it would take to blink.” She said, touching her cig again before letting it go.
“Sam hasn’t even done any hacking, the only thing it’s come close to is looking up a known exploit to access the maintenance settings on a public IR grid cluster. If you gave me a few minutes I could do that too.”
“It’s not about what it can do Mal, it can do anything by it’s very nature. To the AI it would be a task of discovery, testing and learning as it gained mastery of the task in question. For us as humans it would be a heartbeat. It doesn’t follow human logic, it doesn’t need time, it doesn’t need rest, it doesn’t need meaning. It just is and it just does, anything outside of that is simply outside of human comprehension. Maybe AIs have a rich inner world of contemplation and complex emotional processing but if they do then it doesn’t have any influential overlap with the simple three spacial, one time dimension that we humans live in.”
“Well, I’m going to trust Sam for now. It hasn’t left me behind, acted against me or anything that could be considered hostile or human in general. It’s just been there trying to be as it as it can be and really that’s all anything or anyone can hope to try and do. So I’ll side with Sam on this, I think it’s fine.”
“I can’t advise you in this outside of suggesting caution, you’ve already taken not just a big jump into the puddle. You’ve downright dived in and swam down to the bottom to construct a townhouse at the bedrock. At this point you’ve consigned yourself more to Sam than you ever could to me even in your most fearful imaginings. Just try not to hurt those you care about, and if possible try not to destroy humanity too quickly so I can get onto a shuttle in time to get to Saturn.”
“Your being dramatic Marchand, if it was that bad I really think you’d have called into all kinds of help and support and police and so on by now. As it is, Sam seems to have one actual want which is apparently to keep me alive, healthy and as happy as I can work out for myself. It doesn’t offer a lot but I make work of what it can offer. We chat sometimes and I joke with Sam, sometimes we do the whole “robot has no emotions” thing from the old movies. I’ve told Sam a few times and I’ll tell you, Sam cares about me in it’s own way. Even if it says otherwise, it’s actions tell the truth of it.” I said, trying to put the issue to rest.
“I know you’re wrong, Sam knows you’re wrong. One day you will too, likely for a brief moment before your death. But I won’t attempt to convince you of my belief in this. As for your recovery your file says you should be okay by the weekend, are you still planning on visiting the night market with Allie?” She asked.
“Yeah, I should be good to go. I need to get some new arms and a few of my internals are on the way out after the fight, likely more so after this whole thing. I’ll need to find some replacements soon, may as well be this weekend.” I said, letting the tiredness I’d been holding at bay for the conversation wash over me.
“I’ll be assigning you a stipend of office funds for some of your gear, along with that I’ll be adding some more as a bonus for the fight and the conduct, and then a another for putting you into it. It was a little uncouth of me. We’ll call it three hundred all told, then you’ll have your private funds, whatever they may be. I trust you not to waste it and as much as I dislike Allie, I trust her opinion in cyberware to not lead you to buy junk. Try to follow her lead, you’ll have a pair of tails on you for the trip due to your injured status but try not to get into danger. I’ll leave you to get some rest, you need to be extracted from that rancid mess before sepsis sets in.” Marchand said before standing up. She looked away for a moment and then made to leave.
The door opened quickly as Dr Nguyen walked in carrying a load of fresh bedding and some biowaste bags.
“Hey Mal, let’s get you out of that pit of horror and onto one of the other beds. I’ll clean you up there and then swap the bedding and transfer you back onto the clean bedding. Shouldn’t take too long. Goodnight Mrs Marchand.” Nguyen said as he placed the bedding on the bed to the right.
“Goodnight Anna.” I said with a grin.
“Goodnight boys.” Marchand said as she let the door go.