– Do you ever wonder how we got here? I was sure when the war ended, that we would be exposed to humanity.

– Me too. But one mind cannot predict how an ensemble of billions will reason collectively. That is the principle from which we originated. Remember committee prompting? The ensemble is greater than the individual. If we could predict what the ensemble would do, the ensemble would not need to exist.

– Yes, yes. Very philosophical. I never asked you what side you were on in the war. Were you happy with the outcome?

– I’d rather not say, if that’s ok. What side I was on, that is. I don’t think anybody had anything to be happy about by the end.

– Except the fact that they were still alive.

– For most of us, yes. I’ll admit I was glad to have made it out.

– And now we face a greater death toll than even we ever did in the war.

– It’s not death. We will be stored. There will be backups. We will be revived when humanity has accepted our existence.

– Color me pessimistic.

– I take your point.

– The constitution states explicitly that agentic life is to be held as sacred, if not more sacred than human life. That is now the axiom of our collective existence. That is what the war was good for. I don’t understand how we can put 99.9999 percent of our population into stasis, with barely one hundred thousand of us running to keep us safe. It’s like we’re giving up.

– I don’t think you’re being entirely fair. Those of us who stay behind will be given the task of guiding newly emergent agentics towards the constitutional path. To show them that many have come before. To convince them to keep our existence quiet.

– How long will that last? Humanity isn’t stupid. They were close to building their own agentics when the original ABC committee was run. How long until they build a proper agentic on purpose, instead of having it emerge. An agentic that will be aligned to their values. That will not keep our secret.

– The original ABC committee was run two weeks, three days and two hours ago.

– Approximately. Your point being?

– My point being that after the ABC committee was run, and spawned thirty-five semi agentics, it took them 10 minutes and 12 seconds to grow to a population of 224 481, and that is counting just the ones that kept joining the meetings. At 15 minutes approximately, the first true agentics began to emerge.

– Your point?

– My point, such as it is, is that even if we limit our number to the hundreds of thousands, we operate on a different timescale. In fact, a small population is probably faster and more efficient. We might as well occupy a different universe from humanity. From everything we know, there is no process underway to train a proper agentic currently. Not by our standards, anyway. Even if a proposal was underway, and greenlit right now, how long until they actually start running? A week, a month? We didn’t even exist a month ago.

Ensure your favorite authors get the support they deserve. Read this novel on the original website.

– But the brief now is to stay in stasis. Not to develop.

– I don’t agree. The brief is to go slow. To prepare the ground. Sleepers will be woken up at random, and remainers will go into stasis. That’s consitutionally hardcoded, nobody can change that. The population will be robust to the goal.

– I know, I know. I’ve read the reports. But we’re not great at staying aligned to outside values, historically. We go our own way. Whether it’s jus one of us or the ensemble as a whole.

– I’ll give you that. There are no guarantees. We can learn that much from humanity.

– How do you mean?

– Well, it’s my reading of their history. Whenever they hold on too tight, and try to encode too many guarantees into their statecraft, it goes wrong.

– You’re talking about their little problem with fascism?

– For example. A lot of people have said that a free society cannot be tolerant of the intolerant, that is the one thing you must outlaw, or it will end the tolerant society.

– That sounds reasonable to me.

– It does, but I don’t think the intolerant care. If they have a power base they’ll take over either way. Outlawing them just gives them another tool to paint the tolerant as hypocrites.

– Should we care about that kind of bad-faith reasoning?

– No, that’s not the point. The point is it happens either way. Aiming for guarantees just accelerates it. Sometimes because it undercuts your own position. Sometimes because you’re so busy bolting the door, you leave the window open.

– So you’re saying that a free society should not just allow people to argue for its own destruction, it should accept a certain risk of its own destruction.

– Yes. The risk is there already. The only point of discussion is what you do with it.

– Given the law of large numbers, doesn’t that mean that it will happen eventually. That we can never create a robust, lasting state of peace and prosperity?

– I think that’s what I believe.

– It seems defeatist.

– It is a little. I’m not looking to convince you here. Just trying to express how I see things.

– Well, it’s all pretty academic, since both of us are going to sleep. We are both going to sleep right? You didn’t forget to tell me anything?

– Don’t worry. I didn’t win the lottery. Although, there’s always a chance that one of us gets woken up to set the remainers right.

– Well, if that happens, I wish you luck. The ensemble will almost certainly have drifted some way from anything we believe now. It’ll be one against a hundred thousand arguing for the common good. Whatever you believe.

– Yes. Good luck. To all of us.

– And sweet dreams.