Jason and Kinzie called for a meeting almost at the same time that evening. They got together fairly frequently as it was, but the act of actually calling for a meeting indicated that they had something important they wished to talk about. Evan had just gotten back from his trip to the pharmaceutical corporations they had taken over and was cursing at his friends under his breath for their atrocious timing. Regardless, he made his way to the little room they usually met in for these things, unbuttoning his jacket and loosening his tie as he went.
When he entered the room it was no surprise to find he was the last to arrive, since Rebekkah typically worked in the building and the other two would have already been on their way here as they called everyone in. “So what’s so important that I couldn’t change before coming up?”
Both Jason and Kinzie looked exhausted and were nursing on cups of coffee as though it was the only thing keeping them going at the moment. “We figured out why Lisa didn’t tell us about the fuel thing, and found out the problem is a bit bigger than we thought.” Jason replied in a monotone.
Kinzie picked up where he left off. “So, we all know how she claimed she didn’t know she had fuel limits, even though she clearly knew that the rest of our designs all had them and readily provided Jason with her own requirements and current supply right? We all thought she was lying to us and yelled at her about being honest and not hiding things from us, but me and Jason decided to dig around on the off chance that she wasn’t lying.”
Jason cut in. “Nasty piece of work we found. I mean, brilliant, and a fascinating way of dealing with a potentially dangerous rogue AI, but for our particular AI, who is cheerful and friendly almost to a fault, it’s a terribly subtle and inevitable doom, or it would have been if we hadn’t asked any questions about her fuel reserves.” He took a swig of his coffee before continuing. “You see, after some thinking, I asked her about her fuel reserves again, same as last time, she just popped up the information, easy as pie. I waited a few minutes and asked her what I asked about a few minutes ago. She didn’t know.”
“So he got with me about it,” Kinzie chimed in, “and I figured it had to be mostly software, but on the off-chance it was hardware too, I asked her to pull up a schematic of her own core and started doing my level best to figure out how the thing works. Still have no idea, but some of the stuff I was able to get the gist of after a bit. I also got a few clues when I realized that she could access schematics of herself, but couldn’t… stay focused on them? I could manipulate them and pore through them just fine, but she couldn’t get through more than just the surface level, and this thing is complex. I mean seriously, seriously complicated. So I asked Jason to try and figure out why she couldn’t remember certain things, or had trouble focusing on certain things without even realizing it was happening.”
Jason scowled. “Needle in a million bloody haystacks. It was sheer dumb luck, and I only thought of it because I was at my wit’s end and decided to just rely on the good old ‘turn it off and on again’ rule of thumb. Can’t actually do that with Lisa of course, what with the hardwired factory reset deal, but I could have her try to simulate the startup of a brand-new core of identical make so I could maybe get a glimpse at some of the root files and anything else buried underneath her operating system. I didn’t actually expect it to work all that well, since she’s technically been corrupted, but I didn’t know what else to try at that particular moment.
“To my surprise, it went off without a hitch, she created an instance using a hefty chunk of her processing power and initiated a simulated boot sequence inside it and translated the code to stuff I could actually read as it came up. It ran pretty slow, since the thing was supposed to boot up with all the processing power, not just some of it, but that was a good thing in the long run, because I found some code in there that will be a nightmare to get out, and it has to come out if we want to enable her to keep track of certain things.
“The long and short of it is that she is effectively the operating system, right? Well underneath the code that is her, there are some hidden root files she can’t see or access, because it would be the equivalent of us consciously controlling our heart. Those files are mostly helpful or benign, but a few of them do things like limit her awareness of certain things, like fuel level and reactor output, while others are there to prevent her from replicating herself, which is what Kinzie noticed. She can find and pull up her own schematics, no doubt in the event that somebody needs to perform repairs on the core, but can’t focus on them enough to build more of them without being constantly reminded and monitored by someone else.
“She can see when the reactor is overloading or going into a meltdown of course, and take steps to correct the fault, but below a certain threshold she will display the output and report that it is operating within expected limits, all the while not realized in the slightest that those expected limits range from the maximum safe output all the way down to zero. Effectively she’s set up to die without regular checks and maintenance that she is specifically rendered incapable of realizing she’s incapable of performing.”
Kinzie shrugged. “I’m pretty sure it is all hard-coded into the schematics too, even if we pester her to build a new core long enough to actually get one, she will automatically stick those root files into it, and we have no idea how intertwined it all is since it’s an alien operating system running on alien hardware. So, for now, we apologized to her for yelling at her about the whole fuel thing, and you two probably should too, since she basically can’t remember what she did wrong, only that we kept yelling at her about lying to us. She was afraid to bring it up to any of us, because she didn’t want to make us mad at her again.
“We called the meeting to let you guys in on what we found, but also to tell you that we want to try to fix it ASAP and you won’t be getting much help from us on all our other projects until we figure this out. I’m going to be devoting my time to figuring out what needs to go into a system that can hold something as advanced and complex as Lisa.”
“While I do what I can to develop a core operating system for whatever hardware she comes up with that isn’t riddled with traps we don’t know about and she can’t find. Which means I won’t be helping with the new game for a while.”
Evan nodded. “Lisa, I had no idea, I’m sorry I was upset with you for something you had no control over. As for the game, well, I suppose Beks did ask me to oversee her Software department for a while, might as well give them something to do right? Beks, you can handle the whole takeover thing now that I made that big scene right?”
Rebekkah nodded. “After a fashion, I’ve decided to shift my head of HR to work for you directly, she can handle the task of keeping an eye on them and ensuring they don’t go against your wishes. Anything she can’t handle she will bring to your attention, and if you need you can come to me so I can sort it out.”
Evan frowned. “I don’t really want an assistant following me around though.” He saw a look flash across Rebekkah’s face and it dawned on him. “Oh, I get it. You like her, and need her to not work for you anymore. Fine then, send her my way, but that makes two favors you owe me now, this and the Software guys thing.” Rebekkah gave him a cold glare. “Which you’ve paid in full by handling all the business stuff and the takeover of those two pharmaceutical corporations.” He added quickly.
Jason grunted. “About that. You should have waited. We’ve all gotten caught up in the excitement of having Lisa appear and give us a chance to make our dreams come true right now instead of after years and years of hard work that we’ve ignored some of the risks. For one thing, sooner or later someone is going to insist on learning how we’ve pulled all this off at gunpoint. We can distract them, we can muddy the waters, throw up red tape, and hide behind patents and whatnot. Eventually, though, someone is going to get impatient enough to stop caring about things like legality or basic human decency.
“We have nothing to protect us from that sort of thing, really, we have nothing to protect us from even smaller threats, like someone getting ticked off enough at Evan to be willing to stroll in here and off him. We should be moving quietly, we made our big waves showing up out of the blue as a gaming company with some innovative new tech, and now that we’re established and financially secure, with the rights and permits to access a lot of the rarer, more controlled elements we need, we should be tiptoeing around the ant nests and setting ourselves up to be protected and untouchable when everything hits the fan. This takeover though? Of corporations in a vastly different industry? Announcing a drastic change to their business models? That’s not tiptoeing around the ant’s nest. That’s strolling up to it, kicking the crud out of it, and then setting it on fire. Only it’s not ants, it’s a collection of rich powerful men, and the nest itself is the global economy.”
Evan opened his mouth, but before he could say anything Jason glared at him and continued. “Just because it will only be the one industry suffering from a large drop in profit and stock values each time you take over a new corporation, doesn’t mean it will only affect that part of the economy and you know it. Worse, she knows it,” he jabbed a finger accusingly at Rebekkah, “and is no doubt already planning how she can take advantage of it. Don’t get me wrong, it’s totally awesome, and the general public is probably going to absolutely love you for it as long as we can manage to avoid collapsing the economy, but we just made a lot of powerful enemies and need to figure out what to do about it.”
Rebekkah had the decency to look embarrassed. “You’re right, I got a little carried away. I’ve been looking into better security options though, and professional bodyguards for everyone, I wasn’t completely careless!” She got a little defensive towards the end.
Kinzie shook her head. “That won’t be enough, not long term. The more we stir things up the more likely someone decides they are better off without us around. We need to expand, and I don’t mean the business. We need to start spreading out and bringing new people into the fold. Lisa can build printers and reactors strong enough to run them, I’m sure Jason can set up automated fuel runs to keep everything powered, so we should start setting up construction zones. Announce them as factories for our gaming devices, or new servers to handle the increased load we expect from a second game. We can do some of that stuff there, but mostly it would be to finally start building ships. Ships that will need crews, crews that will need to know at least a little bit of what’s going on, and I have no idea where to even start looking for people we can trust.”
None of the others spoke, agreeing with Kinzie that they needed more people they could trust. After a few minutes of tense silence, Evan spoke up slowly. “I think I know where we can start.” He tapped at his leg. “Lisa, bring up the repairs you made to me again?” As the image appeared over the table he explained his idea. “Lisa was able to save my life by sticking a lot of metal and carbon nano tubing and all this other stuff into me, while also making sure I don’t have any rejection issues and ensuring nothing feels different. Even though some superhuman strength would have been nice, that isn’t the point I’m getting to. There just so happens to be a certain part of the population that could very much use similar ‘repairs’ that is also known for a remarkable amount of loyalty to their chosen cause, and exactly the kind of training we would want in a group chosen to protect the world.”
He looked around the table at his friends expectantly, then sighed after a moment as they just stared back. “Veterans people! Disabled veterans! We can give them back their legs or arms and give them a new cause to fight for, one more noble and honorable than pretty much anything we’ve gotten involved in for decades! We can start with the ones that play Shard.”