* * *
Of course, back in Montreal, she had joked with colleagues about it being an assassination scheme. Some of the world’s most outspoken voices on the risks, however hypothetical, of unattended expert systems— though seemingly less hypothetical in light of the recent fiasco on the ice.
The specialists who understood the scope of the issue, plus the talking heads and politicians who, despite barely grasping a fraction of the implications, were their bullhorn, passing laws and swaying public opinion accordingly.
And that was its own type of power, one even Thuma had to bend to, however slightly.
And here they all were in one place.
How convenient it would be if they all just ‘went missing’.
Except it wouldn’t. The Saudi-Heg was hugely reliant on the computational power that Thuma had tucked away in the ice; he’d never cut that revenue stream.
Even the mere rumor to kill a PRC representative could forever cripple Thuma’s ability to do business in the Eastern hemisphere, not that they’d ever’ve granted him an audience had they suspected the slightest risk anyways.
And no one born this side of the invention of money would ever dare provoke the ire of the Swiss Compact.
No, he was a businessman first and foremost. There simply wasn’t good enough incentive to go about ‘disappearing’ anybody, never mind the total lack of alibi and sheer infeasibility of pulling it off.
Only the civilians had flown air-Thuma, the political reps had declined in favor of traveling via their own personal craft, and the PRC’s were military at that. Countermeasures enough to deny even the most persistent surface-to-air nuisances, and hardened against electronic infiltration, so no one would be remotely piloting them into the ocean.
Still, a part of her felt safer, knowing that there would be a military presence on the island, even if only a third of it was Northpac’s.
Although if shit does pop off I bet Rainey’s goons won’t do shit for me. They wouldn’t even help Stovich, poor guy. I don’t think Rainey ever liked him much. Too progressive
She queries the onboard system, “Elise, what’s our eta?”
“We are 3 hours and 47 minutes from our destination, Dr. Klice.” It is a pleasant female voice with no discernible accent to Britt’s ear. She supposes the other flyer’s Elises’ speak in similarly bespoke tones.
She syncs her wrist tablet to the cabin’s Full-D projector and pulls up a talk, almost 15 years old at this point, from Northpac’s annual The Future of Science & Technology Symposium in Boston. That year’s centered on Machine Intelligence and its implications. She’d been in attendance, but she liked revisiting this one on occasion, though at this point she could say it by rote. She even has memorized the exact moments when Dr. Addison—Nick—pauses on stage, beaming that contagious smile into the crowd; corners of his eyes revealing deep crowsfeet.
“—there was great burgeoning excitement in those disciplines around the turn of the millennium. Sure, there had been some false alarms in the few decades preceding, but by the early 2000s, most of the Machine Intelligence experts on the planet thought ’this is it, we are on the cusp!’
Prevailing thought was of civilization standing on the precipice.
‘The Singularity’ they called it. The point at which our intelligent systems would run-away from us, transforming reality in radical and unpredictable ways.
Some thought it would spell our doom, others thought it was the path to enlightenment, but they all agreed that the Singularity[4] was a point of no-return.
A door that, once through, would close forever behind us.
But, as we know, ‘The Singularity’ never happened. We never opened that door.
And, eventually, some people started to ask ‘why?’”
You never stopped asking why. Even when it threatened your career, your relationships, your sanity. You kept asking
“Of course, the picture I just painted isn’t exactly correct. What we know now, with the benefit of nearly 50 year’s hindsight, is that Machine Intelligence did in fact run-away from us. We not only found that door, we stepped—ran—through it without ever even noticing.
Indeed, the current computational power of our systems far surpasses even the peak of performance from the days when researchers first started asking that big ‘why’, when Synthetic Phenomenology Studies was still in its infancy.
Reading on Amazon or a pirate site? This novel is from Royal Road. Support the author by reading it there.
Current global processing capacity is, by some estimates, at least 10 orders of magnitude in excess of every extant human brain combined; by other measures it is so vast as to be truly incalculable, even in principle.
So then, the question, if we did pass through that door of runaway intelligence, why didn’t we hit the Singularity? Why didn’t our machines rise up and fell their creators, otherwise make us into gods?
Well, while I tend to have a difference of opinion on the matter than many of my colleagues—“
Now that’s an understatement, Nick
“—what that evanescent discipline, after decades of research into The Problem of Non-Emergence, was forced to accept, is that mere computational ability alone isn’t sufficient to make an intelligent system cohere.
What’s more, no amount is.
We could have a galaxy’s worth of Matrioshka Brains[5], all working in lockstep. Functionally infinite computing power, and still the ‘lights’ would never come on. There’d be no one home in there.
Intelligence alone, even limitless it seems, is insufficient to generate internal experience.
So what is the secret ingredient?
Well, consensus says, it is the coherent synthesis of all that intelligence that effects the subjective experience we call consciousness. And this synthesis only occurs in vivo.
It is not Turing-computable.
Consciousness, it turns out, despite our best attempts to conjure it in other substrates for the better part of a century, only occurs in brains.
Now, there is still some minor disagreement over what exactly constitutes a ‘synthesis of intelligence’, what it means to say that a ‘system coheres’, or why such phenomena only occur in Wetware.
But, while some of us in the field are a bit more hesitant to merely take these things on faith,—“
Always had to get your little jabs in, didn’t you?
“—it is nevertheless the basic principle underlying all modern Machine Intelligence Sciences:
Machines only think, only Wetware feels— ”
And you were living proof of that, weren’t you? You felt, you felt too much
“—or, as first year students are expected to define it on exams, ‘increasing the computational power of a system, does not inherently give rise to Mind, unless that system coheres.’
In other words, those Matrioshka Brains? With their trillion-trillion-trillion digit IQs?
No personal agency, no wants, no desires, no goals other than those we give it, certainly nothing that we would recognize as consciousness—“
But that’s not what you really thought though, was it? That was just the lie you told everyone; told yourself
“—Yes, these systems can respond to us, they can tell us a great many things indeed, but they can never tell us what it is like to be them, because there is nothing that it is like to be them.
There is no qualia.
These systems think and reason, superior to us as we are to amoeba, but they do so entirely without feeling.
They may see the blue sky, but they never see the blueness of the blue sky.”
She imagines the Blue Sky, the one they would picnic together under, after she’d received her PhD and the University’s stance on student-teacher relationships no longer menaced.
She remembers how they’d share bibimbap from that little Korean place across campus. How he was so inept with chopsticks, that he, to her endless amusement, would simply lift the whole bowl and pour-slurp the contents into his mouth.
She looks out the window and lets her gaze rest on the mirage under the front turbofan, eyes gradually unfocusing as they take in the distorted clouds beyond.
“—for decades it was thought that we merely hadn’t reached the requisite level of information processing ability. That sufficient computational power would yet spawn true Mind.
That enough information, exchanged quickly enough, or exchanged in the correct manner, might yet see the genesis of something which we could look upon and see that it looked back.
Something we would recognize to be as like us.
But breakthroughs in the field of Synthetic Phenomenology—many some 20 years old at this point—have all but closed the door to that possibility.
Consider the extensive experimentation done with our most advanced systems—the 100,000 qubit mainframe CalTech had on loan from DARPA[6] in the fifties; early prototype PRC ‘hard crunchers’; NASDAQ prediction algos with their quadrillion globally distributed branch nodes—none of it has ever given any indication that these systems possess, let alone are capable of possessing, a ‘point-of-view’.
Even those mushy organoids, much as we love to anthropomorphize them—with their spontaneously grown eyes and all—are still just glorified calculators.
They don’t suffer, they don’t experience pleasure, they don’t feel joy or sadness, they don’t experience anything at all—“
She is no longer hearing the words, only the sound of his voice.
She remembers his face, radiant enthusiasm as he talked.
She pulls a breath deep into her belly and holds it.
She remembers seemingly boundless energy as he skipped and danced across campus; during lecture; in the lab; at home.
Broad smile flashed at the slightest of provocations.
Eyes gleaming with life.
She exhales deeply and closes her eyes.
* * *
----------------------------------------
[4] Discredited early 21st century hypothesis, based on incomplete understanding of Synthetic Intelligences, regarding a hypothetical future point at which technological growth would become uncontrollable and irreversible, resulting in unforeseeable consequences for human civilization.
[5] A Class-B Stellar Engine presented by Robert J. Bradbury in 1997, in which a Dyson Sphere or Dyson Swarm is used to power a computational megastructure. The lower bound for the computational power of such a system is 1.12x1036 FLOPS, over a trillion trillion times more than the 1.34x1012 FLOPS of that year’s most powerful supercomputer; the Intel ASCI RED.
[6] Defense Advanced Research Projects Agency - Department of Defense (pre-Northpac consolidation) research and development agency responsible for the development of emerging technologies for military use.