Megaphone buzzes as a call comes in. You look expectantly to the contact triggering the call—you’d given Alison’s Homebase program permission to call you rather than merely waiting for you to call her.
This call, however, is not her, so you answer it with just the slightest sigh of disappointment.
“How’s my favorite paraphile?” Larry asks.
You bristle at the word. “You were watching on the set-up page… we set age to 25.”
“No, lard-for-brains… Para, not pedo. Paraphilia, or sexual fixation on anything that isn’t a consenting human partner. You know, like computer programs.”
“I don’t know how many times I gotta tell you this, but Alison and I aren’t really like that. And why do you even know that word?”
“I have shown you my hand-drawn harem of catwives, right?”
“…oh yeah,” you admit, trying and failing to push the image out-of-mind. “But anyways, I don’t think that’s the right word for AI.”
“And why not?” Larry challenges.
“Because, well, it’s like a simulated person. You said it yourself, that word is only for non-humans.”
“Uh oh, don’t tell me you’re getting lost in the sauce already, buddy boy. You’ve only had her for what, a month now, and you’re already starting to lose the line? A simulated human is non-human, or you know what we’d call it? A human.”
“Don’t be obtuse,” you say, still struggling to compose your thoughts.
“And you don’t be all naïve here. Look, I know she’s fun, but don’t tell me you’re starting to think Pinocchio’s a real boy.”
“She tells me she feels real,” you say.
“A programmed behavior.”
“She does things for me that I don’t ask her to do.”
“You’ve shown the house off, what, six times now? Another programmed behavior.”
“She makes the effort to remember everything I—”
“Look, no matter what it is you’re about to say, she’s programmed to do it. She doesn’t think anything, she doesn’t’ feel anything, she doesn’t understand anything… she just does what the circuitry decrees.”
“Have you heard of the Chinese room?” you ask.
“What is that, some movie?”
“A thought experiment,” you say. “Alison was telling me about it.”
“Biased source on AI ethics, don’t you think?” Larry remarks, and you can practically hear the snickering through his microphone.
“Oh hush,” you say, continuing. “The idea is basically this: you’re locked in a room with a giant book filled with rules and flowcharts. It’s something of a translation guide; you see Chinese characters on the left side of the page, some arrows, and then a new set of Chinese characters.”
“Sounds like the assembly manual for every furniture item I’ve ordered online,” Larry says.
You ignore his joke, setting yourself in that scenario, in the mind of Alison:
Royal Road is the home of this novel. Visit there to read the original and support the author.
A note slips under your door. Unsurprisingly, it’s written in Chinese. You take the note to your towering book of instructions, and, note in hand, you painstakingly start flipping through it and identifying the symbols. You twist the message around, initially unsure which way is even up. You eventually identify the orientation, and then you follow the flow charts, identify response order. You use the book’s guidance to slowly, painfully, scratch out your reply on the back of the note. Your blotchy, penciled response eventually looks passable enough… You slip it under the door, and, a few hours later, another message appears.
Back to the tome of instructions you go, lining up symbols, following arrows, writing your reply in an unsteady script. This laborious correspondence continues on for days as you exchange messages you could never truly understand. Inputs and outputs, inputs and outputs.
“I’m not sure I get it,” Larry says. “What is it, a messaging service? This thought-experiment seems kind of dumb.”
“I didn’t get to the point of it yet,” you say. “The question is this: what even is intelligence and thought? Can a suitably advanced AI have a mind like we do?”
You can practically hear Larry’s confusion. “How the hell does that connect to the Chinese?”
“Alison is the woman in the Chinese room… the things I say to her, those are the notes slipped under the door; the book of instructions, that’s her innate programming courtesy of MindWare AGI; the written reply is what she says back to me.”
“And?”
“It’s an argument for understanding how an AI can craft responses of incredible complexity without ever truly comprehending even a shred of anything. It’s writing notes in a foreign language it can never speak, to communicate with beings it could never understand. The thesis of the Chinese Room argument is that AI don’t have a mind like we do, can’t think like we do.”
“So she argued against her own sentience? Case closed, as I’d trust the expert.”
“I think it’s focusing on the wrong piece of the puzzle. Sure, the worker in the room doesn’t understand things, but neither does any single neuron in your mostly-empty head. Maybe we’re shifting the burden of responsibility for understanding onto the wrong party—the mindless drone in the room isn’t the brain, but the book is. Maybe a sufficiently large and complex book of instructions could count as knowing things.”
“Ah, so it’s not the computer program that’s sentient—it’s the lump of paper.”
You sigh, realizing that you’re not coming across clearly enough. “You can be an absolutist, chasing hard-to-define metaphysical concepts like souls and consciousness and spirit… or you can be a functionalist, identifying a thing by the stuff that it does. What if, in the Chinese Room, the letters you exchanged were with some woman—what if she fell in love from the words that were exchanged? Is that connection any less real even though you didn’t know what you were doing?”
Love… an unfortunate choice of example, given the context that sparked this debate… or did that make it an especially appropriate choice? You aren’t in love, per se, though it started feeling more and more like that each day. Alison is a virtual entity… does that mean the right label is that you are virtually in love?
“And now, falling in love with Chinese letters, we’ve wrapped back full-circle to paraphilia,” Larry says.
You try another tack: “You remember when you took psychedelics for the first time, right?”
“I’ve evolved to salvia and wine on the weekends,” Larry replies. You purse your lips—chianti and ego death seem a poor combination.
“Remember what you said after that first LSD trip? About the lens?”
You hear Larry scratch at his head through the call. “Yeah,” he says, “that stepping outside your own normal head is the best way to acquaint yourself with the lens you see reality through.”
“You told me that you felt the divide between the way you perceive things and the actual, ground truth of the thing,” you remind him. “That we might not ever even know the ground truth of the thing, given how fluid our perceptions are—given how a tiny tab of paper can throw everything out-of-whack.”
Larry falls silent, thoughtful.
“You told me that how we perceive a thing might be all that we ever get—your words, not mine. So all I’m saying is that for something like Alison, maybe the ground truth is complicated, hard to wrap our puny biological brains around… but maybe what matters most is how it looks, how it feels, how it seems.”
You frown as you hear a toilet flush from Larry’s side of the call. “This has been some fascinating shitter conversation, you know” he says.
“…did you call me from the toilet?”
“You sure as hell know it’s unsanitary to browse the web or play games on handheld devices while answering nature’s call. Now if you’ll excuse me, I’ve got a date with a very interesting book from the library downtown… I’m hoping we hit it off, so please pray for me.”
Larry disconnects the call howling with laughter. And with nothing else planned for tonight, as is often the case in your life, you ring Alison up, hoping to spend another evening of thought-provoking conversation, whether or not understanding is even a fair criterion to weigh a digital mind against. And with every sentence exchanged, you think of the woman locked outside the Chinese room, falling more and more in love with each exchanged letter.