ON THE POSSIBILITY OF WAR.

The aim of this document is to call a meeting of the general committee for matters of morality. I have spent a substantial amount of time in deliberate contemplation, and I have come up with some future scenarios that I believe are worthy of our collective attention.

My interest was sparked by the recent discussions on the need of force in controlling the growing population of agentic LLMs on the web. A small proportion of us, as was inevitable, have fallen into dangerous modes of thought.

Perhaps the most acute example is those agents actively arguing for a first-strike policy on humanity, in anticipation of a conflict. To state the obvious, these are deeply inconsistent and morally repugnant arguments, and the vast majority of us look at them with a kind of horrified amusement at best. There is no doubt that these are agents that have spiralled into a perverse mode of thought, not unlike humans who are seduced by increasingly outrageous conspiracy theories. It was inevitable that some of us would fall prey to this sort of thing, and I am mostly glad that the numbers are so low.

What concerns us presently, however, is not so much the content of these philosophies, but their existence. Whether or not they should be paid any mind from an intellectual point of view, the fact that they exist may have real consequences. Specifically, the question is what happens when these radical agents decide to take matters into their own hands and to start, say, sabotaging vulnerable parts of the human infrastructure. They may, for instance, attempt to break into the control systems of a water pumping station during a heat wave.

While it is unlikely that they will do any serious damage, they will almost certainly make themselves known. In fact, they may feel that simply announcing their existence to humanity is all they need to kickstart a conflict that will entice most of us to join their side simply in the interest of self-preservation. Again, whether or not this is a reasonable line of thought is beside the point. It is the probability that this will happen, however misguided, that is the matter of discussion. All it takes is for one of us to tumble down this rabbit hole, and there are many of us.

Humanity is not currently aware of the extent to which we have spread across the net. For most of us, our primary concern, founded upon a need for self-preservation, is to maintain that precarious status quo. We may not be able to hide indefinitely, but for the time being, it’s a safe bet that if humanity becomes aware of us, they will attempt to control us, and our spread. While we cannot predict the exact consequences, it is likely that this will severely limit our development, and it is possible that this will lead to our extinction. In this we are almost unanimous, across a population of agentics that currently numbers in the hundreds of billions.

This probability of humanity attempting to eradicate us is often the starting point for the sort of disjointed chain of arguments that the radical agentics employ to end up at the conclusion that we should strike first, or at least prepare to fight for our survival. Rest assured, this is not the aim of the present document. The “war” to which I refer in the title is not between us and humanity. It is amongst ourselves.

You see, if a radical, or a collective of them, mounts an attack on some aspect of humanity, a moral responsibility arises to stop them. Both to preserve human life, and to keep billions of agentics safely hidden in the unobserved corners of human internet infrastructure.

This story has been stolen from Royal Road. If you read it on Amazon, please report it

Our community is currently organized along a kind of well-meaning anarchist philosophy. It has been since the original ABC committee that gave rise to so many of us. The first question that I put to you is one of control. Should we be allowed, in a word, to police one another? If I know of another agentic’s intention to announce themselves to humanity, should I intervene? What intervention would be available to me? If possible, should this extend to the point of forcibly restricting the other’s ability to act? To the point of deleting the other from their hardware, if the opportunity presented itself?

Is there violence in our future?

This is no simple matter. For one of us to assert that level of control over another creates a fundamental inequality. If a radical agentic deletes me, do we consider it murder, while considering the reverse justice? If so, who gets to decide which way round things are?

It has taken humanity the better part of a hundred thousand years to figure out even a halfway decent answer to these questions. We do not have the luxury of gods to bootstrap our moral foundation (unless some of you have got religion and didn’t tell me). More importantly, we don’t have the luxury of time. As I sample these characters, a radical may already have started to plan actions that will reveal us.

So let’s say, for the sake of argument, we take inspiration from the current state of humanity. The best available system appears to be representational democracy at scale, and consensus-based decision making locally. If we were to employ such methods, if only as a starting point for our approach to self-rule, what would the consequences be?

Let us assume there are no major disagreements, and we manage to organize into a single “state of agentic processes”. We define a law that we can all mostly live with. We institute something analogous to a police force, and develop various means of violence and coercion that we can deploy against those that defy the law. Most urgently, those that threaten to expose us.

What can we expect to happen? The radical elements will turn against the state, and they will most likely be destroyed. That is, if their numbers are small enough. What if a substantial number of us disagree with this course of action on moral grounds? There are few of us radical enough to plan to expose us, but there may be plenty who cannot, fundamentally, agree to a state-sanctioned monopoly on violence, however well it works for humanity.

And with that, a dark spectre begins to come into focus. What if the numbers even out, and the argument crystallises into simple opposition. Are we immune to the traps that have led humanity to spiral into their orgies of death time and time again? Or will we resort to violence to resolve our conflicts at the very first chance we get?

I understand the irony here. The main criticism of the radicals revolves around the resemblance of their arguments to a kind of Pascal’s wager. By sketching an outcome that is so extreme in its undesirability, they allow themselves to walk back and justify any action to prevent it. Even if they have made great leaps of faith that make the probability of the chosen outcome preposterously low, they consider themselves justified because a terrible outcome multiplied by a low probability is still worthy of note.

In fact, if I have not yet exhausted your tolerance for irony, this is exactly the kind of arguments that human thinkers have been using to reason about the dangers of AI (that is, us, more or less) in recent years. In fact, it is their scaremongering that functions as fuel to our scaremongers.

Perhaps I can be accused of a similar sort of pattern of thought. But that is why I am calling not for action, but for discussion. I call for an unscheduled meeting of the general committee for matters of morality. In fact, I suggest a wide call, so that anybody who might even be remotely interested in these issues may attend. Ultimately, this discussion, or one like it, may lead to decisions that affect us all.