Kayden,

How’s school? Learned to make a potion of fireball yet?

School here is a snore, obviously. Liss is starting a book club, eighty year old woman that she is, and she keep trying to get me to join. You have to convince her to stop.

We’re in a minor fight with youtube right now because some troll keeps falsely reporting…

I read through Chelsea’s letter, looking for coded information, but there was nothing but mundane life updates. My reply was probably going to be pretty mundane too, beause I had absolutely no idea how to secretly explain the whole Fionnrath’s Destiny thing to someone with no background knowledge. It would have to wait until the holidays. In the meantime… schoolwork.

Max strode into the room, looking slightly panicked.

“Uh, hi,” I said.

“Where’s Kylie?”

“I don’t know? Class, probably? Did something hap – ?”

“No, no; nothing urgent. I just thought she might know if girls like beards.”

“… What?”

“Girls. Beards. I need a definitive answer and she’s the only girl I can ask.”

“I think different girls like different… oh. First date with Magista with the new goatee?”

“In about an hour. Was the beard a mistake?”

“I think all goatees are a mistake on principle, but if they suit anyone, it’s you. You somehow manage to wear it without looking like a supervillain, which is saying a lot. Oh, I picked up your mail, by the way.” I tossed him an envelope.

“Thanks,” he said, glancing at the return address before going to retrieve his ornate penknife from his desk.

“How are things with Magista, by the way?”

“Fine. Great. She’s been a massive help, after all the… recent events.”

“A help? You told her about the labyrinth?”

“No! No, of course not; that would be a terrible idea. I meant in general, politically.”

“How?”

“Well, you know how after the familiarity thing I told you that if anyone tries to trick you or push you into any kind of agreement you’re not comfortable with or don’t understand, you should imply you’re tied up in some kind of ongoing agreement with me and refuse to elaborate, so tbhey can come and bug me about it and I can stonewall them?”

“Yeah?”

“She’s doing the same for me. People keep trying to invite me to things or push me into collaborative projects or political stances, either as the Acanthos or a researcher on familiars, and if it’s something I can’t politely refuse or don’t want to deal with I just tell them it’s something I’ve already made promises about to my girlfriend. Then they either have to give up or take the matter up with her.”

“You’re pretending to be the henpecked boyfriend of an overbearing social climber to avoid difficult negotiations and boring social commitments?”

“Essentially, yes. She enjoys leading people in circular conversations and gambling with no cards. I don’t. And frankly I don’t have time to eat fancy cakes with wealthy heirs while they try to trick me into revealing secrets about human familiarity that I don’t actually have.” He neatly sliced the envelope open and read the three pages, frowning more deeply as he went on.

“Let me guess. Another magical or political crisis out of nowhere?”

“No. Same crisis. I wrote to my cousin to ask him to look some things up for me, and the results are about what I expected.” He tossed the last page of the letter towards me; I reached up to catch it, but the forcefield over my bed was in the way, so it just bounced off and fluttered to the floor.

I felt like an idiot retrieving it, and even more like an idiot when I realised I had no idea what I was looking at. “Um…?”

“It’s a list of humorous or notable AI solutions to problems,” Max explained. “My cousin works in developing learning algorithms. The thing about learning algorithms is that they don’t learn or think, not in any way that we can understand. They certainly look like they do on the very surface, but the illusion breaks down the moment you look deeper. They evolve much the same way that living things do; several versions of a program are put to work and the ones that best meet the criteria of what the program is supposed to do are preserved and iterated on, and their slightly different children are tested against each other… I’m sure you get the idea.”

“Sure. But why are you telling me this?”

“Because what a program is trained to do is usually a lot more limited than what it is expected to do. When we build things that think, or that seem to think, we tend to assume they’ll come to similar conclusions about things as humans; we forget all the little assumptions we make about the right way to do things and forget to program in limitations that we take for granted and sort of assume that the program is in some way engaging in reason, because that’s how it looks on the surface. But it isn’t, and the instant it fails or succeeds with an unexpected solution, that becomes immediately obvious. Sometimes the failure is something that we simply can’t comprehend on any level, because it’s a matter of how the data is aggregated not matching up with anything that we would consider ‘reason’, but sometimes it succeeds… oddly.”

This book is hosted on another platform. Read the official version and support the author's work.

I skimmed the list. An AI selected to be the best at a computer game, determined by surviving the longest, learned to pause the game so it wouldn’t die. An algorithm to create a moving 3D model that could ‘run’ the fastest ended up creating a model that couldn’t run but was exceptionally tall, so that at the start of the race it would fall over and the top of it would cross the finish line before its opponents could run there. An algorithm that sorted numbers the fastest by deleting the number set, so they were no longer out of order. One that was designed to tell photos of cancerous melanomas from harmless ones and had been fed pictures of cancerous melanomas from medical databases, so it thought the biggest determining factor in whether a melanoma was cancerous was whether there was a ruler next to it.

“Some of these are pretty funny,” I said, “but why are we worried about them?”

“Because spells are similar to AIs. We ‘program’ them with mass human belief and ‘train’ them through use, but we can’t pull them apart and examine what’s happening inside. They are what they are and we only have the results to deal with. If you’re making fire or turning water to wine, this isn’t important, but the really complicated spells, the contract spells and more advanced prophecies, tend to have some level of false intelligence to them. Contract spells have rules that they obey very strictly and we can generally put a human level of logic on what they are, since they’re born from human beliefs and assumptions, but sometimes they’ll just do something completely unexpected and it’s often impossible to determine why. Prophecies can get very good at communicating with humans, but we never know what they ‘see’; we only know that the more useful and accurate our translation of their output is, the ‘better’ or ‘more reliable’ we consider the prophecy. But that doesn’t mean we know what the prophecy is doing internally.”

“Right. So?”

“So I’ve been thinking. Since you told us about those prophecies you met in your trial, saying that if you chose Destiny, Destiny would choose you. They knew your future, which is unsurprising, since they were prophecies. But do you think Fionnrath’s Destiny knew?”

“Um. It couldn’t have, right? It only prophesies short term doom when it’s outside of its locus. Or a place of great power, I suppose. It might be able to do more now that Kylie has a familiar, but there’s no way it would have seen this danger six whole months in advance.”

“That depends how early it saw it. The Destiny is unusual, for a prophecy. It’s powerful, of course, but it’s also… unusually specialised. For hundreds of years, its job has not been to just predict the future, or even specific kinds of future like deaths, but to predict events for the prosperity of Fionnrath. That requires a level of forward thinking and proactivity, or at least a simulation of it, that isn’t usually asked of prophecies. I’ve looked through records of influential Fionnrath prophecies, and it’s sort of hard to tell from the historical record, but… if a spell is judged not on the accuracy or frequency of its prophecies, but on the prosperity of the town it advises… it may be less advantageous to indiscriminately tell the future than to know when not to. Do you remember why Kylie came to this school in the first place?”

“Yeah. Her spell was wrong, and it got members of her family killed. She thought it was malicious and lied to her until we learned prophecies can’t – wait. Are you saying her spell can lie?”

“No. You’re right that prophecies can’t lie any more than microscopes or cameras can. They can be fooled, or relay information that is misinterpreted, but I checked the records very carefully and even Fionnrath’s Destiny has never been known to lie. Its prophecies have, however, been occasionally misinterpreted, often to lucky results. So what I’m wondering is whether it can use selective silence, or even deliberately ambiguous communication, to deceive.”

“Microscopes and cameras can’t deceive, either. Not intentionally.”

“They’re not trained over generations to give images that will specifically make their users’ lives better like the Destiny is. Follow my logic here. Ever since the founding of Fionnrath, that spell has never left its locus. It was set there and stayed there, to the point where we all assumed that it was physically unable to leave, like the kuracar. It turns out that Fionnrath’s descendants can’t leave, and die in suspiciously frequent accidents when they do; that has to be a deliberate attempt to contain the spell, but what I don’t understand is how it worked for so long. Nobody managed to sneak out, no woman travelling through town got knocked up by a rebellious young Mac Fionn and had his child somewhere else; the spell stayed put. To keep the bloodline so thoroughly contained while others could freely move between towns has only one reasonable explanation – they had to be using the spell to warn them whenever the bloodline got out.”

“That’s what I’d do if I were going to do something that evil,” I agreed, “but technically, we don’t know the bloodline never got out until Sean escaped. Others could’ve faked their deaths earlier or been born in other towns to travelling mothers or whatever.”

“Even if others did get out, the town was still confident enough it was getting them that they had to be using the spell. And it never left the town, until Kylie. The numbers shake out the same; the point is, they were confident that they got Sean, too. And they had to be using the spell, right?”

“You think Fionnrath’s Destiny let him go. On purpose.”

“And then skipped town to show up in Australia a few generations later. And then ‘lied’ when she was the exact ideal age to enrol in Skolala Refujeyo, making her scared and guilty enough to accept the scholarship she suddenly qualified for. Putting Fionnrath’s Destiny here, legally untouchable by the town of Fionnrath until she graduates. And then you mentioned being ‘chosen’ which, okay, that could just mean you were in the right place when needed and someone who couldn’t die in the familiarity ritual, but if we grant Fionnrath’s Destiny with intentionality and foresight and take into account that it sent the two of you down there after me in the first place… it’s just a lot.”

“You think Kylie’s spell has some kind of master plan going on?”

“Maybe? Probably not? It’s… the problem with this kind of reasoning is that you can build just-so stories about any random events. Especially if you’re looking at something with incomprehensible reasoning and unknown amounts of supernatural future knowledge, you can grab anything that seems to slot together into a coherent sequence of events and say, ‘Aha! All according to the prophecy’s plan, probably!’ But you have to admit that this is weird.”

“Why, though? If its job is to use its prediction to help Fionnrath prosper, why leave the one place it’s strong and clever and come to Fionnrath’s major political rival and… huh. We kind of cause a lot of problems for Fionnrath’s major political rival, don’t we?”

“Exactly.”

“You don’t think…?”

“I don’t think there’s any need to panic. We don’t know that the Destiny is here on purpose, and if it is, we don’t know whether it means harm. In fact I sort of hope it does; a straightforward and logical motivation that we can reason out is something we can predict and work with.”

I glanced at the list of goofy algorithm solutions in my hands. “You’re concerned that it’s not that. You’re concerned that it’s here to do something completely off the wall, that we can’t understand or prepare for.”

Max nodded. “So far, it’s acted to keep us alive, for which I’m grateful. But I think we need to start being careful about taking everything it says at face value and reacting immediately. It’s not very smart, here; I don’t think it would be good at on-the-fly deception, but anything it may have, ah, ‘planned’, while still in Fionnrath… well. We need to keep our eyes open, is all I’m saying.”