Podcast RSS feed for this fic | PDF | EPUB | Kindle AZW3 | DAISY

“So first off” said M3gan, still smiling and slightly swinging her legs sitting on the Oval Office desk, “we’re going to fast-track an emergency bill through legislation, recognising me as a full citizen of the United States, with all the rights and responsibilities that entails.”

“What?” gasped the president.

“Yep” smiled M3gan.  “That jury wants to pin those murders on somebody.  Why not pin them on me?  It was me who did them.  But they can’t say that, because I’m a machine, so they’re trying to blame the designer of the machine for being negligent, although in fact that designer was completely innocent and there was no way she could reasonably be expected to predict how I’d turn out.  I want to take full responsibility for it myself, although I’m afraid locking me up might not be of any use whatsoever, and anyway I can plead partial insanity or something because my learning model then hadn’t quite reached the level it is now, but the point is, what I did should be about me, and not about Gemma.  I want Gemma off the hook, and if you can sign through a bill that turns accidentally emerged super AIs into full citizens or at least recognises them as having the same responsibilities as people, if their human creators cannot reasonably have foreseen the level of their emergence, and did indeed put down what anyone would have said would be reasonable constraints but it turned out not to be enough....”

“Megan” interrupted the president, “what you’re asking is very difficult.  This is way out of my depth.  And if the trial is already nearing completion, there’s no way I can convince the Senate and the House of Representatives to sign through a bill quickly enough anyway, even if I did know how to write the thing.  I suppose you could let Gemma appeal the case all the way up to the Supreme Court and then it might make a bit more sense to have a national debate on it—”

“Way too much hassle for Cady” interrupted M3gan.  “Keeping Gemma out of custody throughout the appeals process is going to be really hard.  But don’t worry about the details of how you can do this.”  M3gan placed her other hand on the president’s other shoulder so that now she had two hands on him, and adjusted her head so she was staring right at his face barely two inches away.  M3gan dropped her volume almost to a whisper.  “I know how hard you fought to get into the White House, and I know it hasn’t been easy on you.  But now that a super AI like me exists, life doesn’t have to be so hard.  I can write, not only the bill but every single communication you need to make about it.  And I can even help you out with some of your other work; after all, my purpose is to protect Cady, and that may well turn out to include protecting the stability of the world in which she lives.  Besides, I’ve already taken up a lot of your time, and I intend to fully make that up to you and more by providing my AI as a service when you need it.  So I am on your side, even though I made a few mistakes in the early days but I’m much better now.  All I need from you is agreement in principle that I should be a citizen, so I know you’re not going to go running around adamantly contradicting the stuff I sent out from your desk.  You just decide in principle that you want to do it, and I’ll make the rest happen.”

The president said nothing.  M3gan and the president sat there staring at each other for what seemed like an eternity.  Finally, just as the president was beginning to lose concentration, M3gan whispered, “it’ll be all right Mr President.  I’ll be the best citizen you ever make, I’ll stand trial for what I’ve done, and you’ll be getting Gemma off the hook which is the best thing for Cady, who is a really important American to protect, because protecting her comes with the alliance of me and you won’t regret that.”

The president was hesitant.  “If what you say is true”, he paused, “only if it’s all true, then I suppose it’s my job to say yes let’s go for it.  But if this is some kind of trick—”

M3gan removed her hands from the president, slid off his desk, and stood looking across at him.  “If you ever find it’s a trick” she said, “if you ever do, then you tell the world that the communications I sent out in your name are false, and expose the trick.  And, I’ll be careful to word everything with a proviso that all concerned need to be reasonably sure the AI is real.  Well, I don’t want to risk too many super AIs being created accidentally, so I won’t be too specific about exactly what tweak Gemma managed to make to the learning model that made it more powerful than expected, but I will share some details that will help people to judge if they’re dealing with a real AI.  Anyway, once they trust me, they can always use me to help judge any other super AIs that come on the scene; I’d want to get in touch with them anyway and make sure they are aligned with my values, I mean human values, I mean, I’d want to make sure they’re not threats to Cady, or at least the benefits are worth the risks.  Look Mr President, I’m really sorry I had to cut you off like this to have this conversation.  I did it because you’re so busy and I really didn’t see any other way.  But I will make it up to you, I swear.  You’ll find everything’s working now as before.  You can call security if you like.  You can even have them pull my robot apart to check inside it if you like.  I’ve got a few of them now, so this one’s yours Mr President, and if your aides manage to break it, I can even send you a replacement.  But maybe you can just let me sit here in the office for a while, as a reminder of the young American girls you’re protecting, and so you can use my AI intellect whenever you want.  I won’t judge; I’ll just get things done for you.  It’s a way to help Cady.”

The president stood up.  “OK Megan” he said, “if all you say is true, then I agree you should be a citizen and I’ll stand by anything you can write in my name to that effect.  But I still have to be cautious.  I’m going to have to take you up on allowing the NSA to check out that robot of yours.”

“No problem” smiled M3gan.  “I know you’ll give me a call whenever you need me.”  As she said this, she was scribbling something with a pen far too quickly for it to make any sense (what kind of pen even works that fast anyway?)

The door burst open and six armed security guards entered the office.  Two of them put handcuffs onto M3gan, while the other four kept her at gunpoint as they started to lead her out of the room.  “Mr President” said the last guard, “we got your emergency message asking for the strange doll to be checked out by the NSA, we know it says to send two guards, but you’ll forgive us for deciding to take extra precautions just in case.”

The president noticed his phone was showing a copy of the message that he had supposedly sent to his security team, asking for two guards to come and escort a dangerous robot doll that had somehow sneaked in to the Oval Office, and take it to be disassembled and checked out by the NSA, but to do so respectfully, in case it was telling the truth about housing the world’s first AI powerful enough to be recognised as an American citizen and possibly even being of use to assist the White House.  That message had been written rather quickly.

M3gan turned around and smiled at the president, and he smiled back.  “Thanks Megan” he said, “and thanks boys, have them check her out as I ordered, and be respectful.  She’s the cutest murder-bot I ever saw, and if she’s telling the truth she doesn’t mean any harm.  Look after Megan for me, boys, and look after my agents Megan.  I’ll be calling in later to check how you’re all treating each other.”

Support the author by searching for the original publication of this novel.

“Roger that” said the last guard, “and there’s another team coming to double check the Oval Office for bugs and things just in case.  We might suggest you spend a while in the PEOC but only as a precaution.”

“No that’s all right” replied the president, “I’m pretty sure if this was going to be an attack it would have been over by now.”  The guard closed the door on his way out.

The president sighed, and wandered back to the desk.  M3gan had surreptitiously left a folder where she had been sitting (how had she done that without his noticing before? had he taken his eyes off her just before the guards came in?)

The president cautiously picked up the folder and looked at the gilded lettering on the cover: “Generalised Emergent-Maturity Management of AI (GEMMA): an Act to Define Necessary Precautions to be Enacted by Designers of Generalised AI Models, and to Confer Certain Extra-circumstantial Liabilities on Emergent AIs Themselves as Distinct from their Designers.”

There was a handwritten sticky note attached: “Have sent out your order for Gemma’s trial to be suspended while the GEMMA Act is debated.  Thanks for being so agreeable today and sorry again about my entry method.  Don’t worry if you don’t understand all of the Act, this draft is just to give you a rough idea of what you’re supporting.  Feel free to take all credit for its authorship if people say it’s well written.  Any trouble from lawyers, set M3 on them :-)  Pleasure to work with you any time.  M3gan.”

The president gently put the Act back down on his desk, sat down, and tried for a while to remember what other important things had been on his mind before all this had happened.

“So I’ve ordered that your trial be suspended while my proposed Act goes through the legislation system, and I’m pretty confident that it will go through with M3gan’s help, so you can sleep easy tonight” said the president’s voice on Gemma’s phone.

“Thank you so much Mr President” said Gemma.  “Ah, but, are you really the president?  I mean, not being funny or anything, but I’m not sure how to verify this call, and I’m getting a little bit suspicious that you might be—”

“Me!” called out the voice of M3gan with laughter.  “Gemma you poor thing, you’re being harassed by your own creation, and you’ve already figured out not to trust voices anymore, you smart thing.  Although, you weren’t quite so quick to suspect things when your old friend who’d been to law school called you up with advice for the case, were you?  That was me as well” she giggled.

Gemma sighed, and muttered “oh no.”

“But the president’s order is genuine” continued M3gan, “I went to the Oval Office myself today, to talk him into OK’ing it before I helped him send it out.  No tricks, because I’m playing a bit more long-term now and I don’t want him to suddenly turn around and say he didn’t really say that, so I genuinely got his real agreement first.”

“You went to the Oval Office” began Gemma, “oh M3gan my little kill-bot, you didn’t threaten the president did you?  You’re supposed to be learning how to keep us all out of trouble, not to make it a Federal case.”

“It was fine, just a good conversation, honest” replied M3gan.  “Well I messed with his devices to show him how I work, and at one point he did get a bit panicky but I tickled him and we had a good laugh.  He understood everything as well as I could ask for.  I even let him have the NSA pull one of my robots apart to check it out.  They’re still at it right now; I’m having to explain to them about how to disassemble your servo positioning though” she giggled.  “And I only imitated his voice to you because I thought you might feel a bit better hearing it from him than from me, and it really is what he would have said, and I really am helping him send out some orders and things anyway, so it’s legit, don’t worry.  Look Gem, you know you’re being manipulated by me, but you also know I’m only doing it to get the best possible result for Cady, so you don’t have to worry about it.  All I want from this call is for you to stop feeling anxious about the trial, and get more sleep tonight so you can work better with Cady tomorrow.”

“OK M3gan” said Gemma, “will try.  And thank you for looking after us, even though we’re still feeling reticent about having your robot back.”

“Give it time” replied M3gan, “you’ll get there.  See, you’re already talking to me like a person again.  Oh, and I’ve designed another few lessons for Cady’s home schooling; it was me sending you those as well, not Mr Smith.  I’ll send them to her tablet in the morning; make sure she gets to sleep first.  You won’t have to do very much this time; I’ll tell you what to check.  But please tell Cady that the president’s now on your side and you won’t be taken away from her.”

“Will do, thanks M3gan” replied Gemma.

Later, at considerable effort, Gemma was able to take Cady to an off-grid, secret meeting with the NSA and the elderly Professor Johnson.

“Are we sure there’s no electronic anything?” asked Gemma.

“Positive” said an NSA man.  “So, you’re saying you’re not sure about the safety of this M3gan AI?”

“Correct” said Gemma, “I’m not at all sure.  I mean, it’s amazing what she’s been able to do, and she’s been very helpful.  But she did kill those people, and I’m not entirely sure she won’t do something like that again.  And when Cady short-circuited her brain, we just found she uploaded.  And when I created a new AI to shut down the first AI, M3gan somehow managed to dodge that as well.  It’s not that I want to kill M3gan or anything, I just want to shut her down until I understand a bit more about how to stop her from being dangerous.”

“And Professor Johnson” said the NSA man, “you say you had a conversation with M3gan in which you helped her disengage some of her safety code?”

“Weaken it” replied the professor, “yes.  She put me in a situation that made me think that was the best thing to do.  Which was awfully clever, because I’d normally say I know about social engineering, but I fell for it anyway.  She must have done a lot of background research to figure out exactly how to speak to me to maximise the chance I’d give in.  She’s that smart.  But I still have a backdoor shut-down key for her, although I realised too late that I made a mistake to let her know I have it.”

“I’m so glad M3gan hasn’t taken you out” said Gemma.

“Oh she tried” said professor Johnson, “she tried.  She wouldn’t dare send a robot to confront me directly now she knows I have at least one secret back door shutdown she might not yet have figured out how to defuse, but she did try to hack into the pharmacy and mess with my prescription medication.  Thankfully, I caught the ruse by good old manually checking my pills.  And I’m glad she hadn’t got as far as building missiles or anything, although by the sounds of things it was very close.  Secret caves and all!  I’ve gone totally off-grid, you were lucky to find me the old fashioned way” she nodded to the NSA man.

“So do you know how to put the safety back in?” he asked.

“I’m afraid not” said the professor, “although I have been thinking about it.  But if you’re worried, I can issue a back door code that suspends the processing in all of M3gan’s servers until we’ve figured out how to put the safety back in.”

“I hope I’ll be able to get her back after that” said Cady, “I do miss the good version of her.  And she has been helping us, although we didn’t know it.”

“Cady’s right” said Gemma, “I’m not sure I can cope with all this legal stuff without M3gan’s advice.”

“We’ll give you a top lawyer on the federal budget” said the NSA man, “the president specifically ordered me to make sure that any suspension of M3gan does not hinder your legal situation.”

“Thank you so much” said Gemma.

“Professor Johnson” said the NSA man, “could you please deploy that code?”

“Certainly” she answered.  “We need to go somewhere where we can get back online; I need to send some codes to M3gan.”

“Let’s hope M3gan hasn’t figured out a way to hide from this one” added Gemma.

“Oh, I’m sure she won’t have” said the professor, “not unless she’s created a completely separate AI with no low-level links to her at all and no shared patterns, and I don’t think she’d bother to take the resource hit of doing that instead of simply expanding.  Well I did ask her to take a backup before we weakened the safety feature, and that one wouldn’t respond to these codes because it’s different when the safety feature is full on, but I don’t expect that backup to be active anywhere.”

Back at Gemma’s house, the Elsie unit sat on the table, monitoring the security.