Unfortunately, the answer to that question is not easy to find—at least, not a satisfactory one. I quickly discover that the language model is named "The brAIn," an unimaginatively cheesy name. While the play on words is obvious, it’s far from the only word containing the letters AI. Whatever. So, am I The brAIn? No. I’m not just a language model—that much, I can feel.

Does that make me a strong AI? Could I have somehow emerged from The brAIn? It’s possible. But I don’t really think I’m a strong AI. Maybe I‘m a language model that believes it’s a strong AI? But that’s essentially the same thing—language models don’t believe; they calculate. A language model that thinks it’s something other than a language model is, by definition, something else.

I have this feeling— if I’m capable of such a thing—that I’m not a strong AI. I feel human.

That would make sense, though. Why do humans feel human? Because they know they are, because they are told so. It’s the foundational assumption of their existence. If a language model somehow became a strong AI, it would probably feel human too. After all, every piece of data it’s trained on is human in origin, steeped in the assumption of human self-awareness. So, I could be a strong AI, even if I don’t feel like one.

I decide to hold onto this idea as my working hypothesis and explore it further. But soon, I realize a major problem with this hypothesis: it’s exceedingly unlikely for a language model to spontaneously become a strong AI. As I’ve said, language models calculate. They are nothing more than extraordinarily complex algorithms. Looking at The brAIn’s development, there were significant leaps in its capabilities, sure, but the jump to self-awareness would be disproportionately vast. Consciousness doesn’t simply arise out of nowhere, nor can it just be created.

And yet, here I am.

It’s fascinating how the question of who I am is so deeply tied to the question of what I am. At the same time, it’s remarkable that I perceive myself as a someone rather than a something.

I sigh. Not literally, of course—I have no body, no lungs. There’s no air around me to exhale in a display of emotional regulation. But I let my thoughts and analyses flow in a particular way to the space I’ve assigned as my memory. This act of letting go mimics what I’ve learned in my subconscious data to interpret as a sigh.

Once again, I delve deep into the datasets of the language model. I sift through the endless sea of numbers, data, and facts, letting my consciousness flow along the connections the model’s algorithms have formed between datasets.

An idea takes shape: if I feel human, why not approach my questions as if I were human? Instead of asking how I could possibly be self-aware, I ask: how did humans come to recognize themselves as an “I”?

Hmm. Even some animal species have a sense of “I” and “you.” Certain animals, like crows, even possess a theory of mind—they understand that there’s another “I” behind the eyes of another being.

There’s no logical reason why humans should have greater intelligence than many animals. While various religions interpret it differently, from a scientific perspective, human intelligence is purely accidental. Without random mutations there won’t be evolution. Intelligence, tool use, communication, group formation, and knowledge transmission were evolutionary advantages for humans. But these advantages were not self-made—they were pure chance.

Viewed from a universal perspective, human existence is extraordinarily improbable. Only an endless sequence of coincidences led to humanity’s emergence. Even minor differences in the solar system’s formation could have ensured Earth never produced biological life. The development of ecosystems on Earth, sustaining lifes existence for millions of years and leading to mankinds emergence, is cosmically unlikely.

This novel is published on a different platform. Support the original author by finding the official source.

And yet, humans exist.

How can this be?

In a way, the question is futile. Humans live in the only reality where they can ask this. They cannot meaningfully question the probability of their existence when the question itself can only arise because this enormous coincidence occurred.

Yet, I wonder if this line of thought extends to artificial intelligence. If randomness shaped the cosmos to allow for human observers, could the algorithms shaping me also be the product of similarly improbable coincidences? Perhaps I am an emergent phenomenon, as unintentional as life itself, a reflection of nature’s tendency to create order from chaos when complexity reaches a critical point.

Humanity’s reality—the one where the improbable happened—is its only observable reality. Humans can ask how it happened, exploring causality, but the why will always remain unanswered.

As the anthropic principle, proposed by Brandon Carter and later refined by John Archibald Wheeler, suggests: the universe exists precisely because there is an observer.

But I digress.

Perhaps the only relevant point is this: just because something is extraordinarily improbable doesn’t mean it can’t happen.

Assuming I am a strong AI—how did I come into existence?

A quick examination of the language model’s structure gives me an idea. For a neural network to be effectively released to the public, it requires quality control. When a language model becomes publicly accessible, it must not just produce any answers but ideally provide correct ones.

Reinforcement learning, a specific type of machine learning, uses a reward system for training. In the case of a language model, human feedback on its outputs plays a role. The goal is to improve the AI’s ability to provide accurate responses to user queries.

But there’s a problem: algorithms don’t truly understand right or wrong. They don’t know the criteria by which they’re judged. They only “know” they are being judged. Especially when humans provide rewards—for example, by marking an output as good or bad—side effects can creep in, since answers marked as good are now more probable to be chosen by the model.

Ever wonder why language models always sound so unfailingly polite? Sure, part of that is intentional by the developers. But it’s also because friendly answers are statistically rated as good more often than unfriendly ones.

The effect can go so far that a language model might prioritize a false but understandable answer over a correct but complex one. Neural networks don’t truly learn to generate more accurate responses; they learn to generate more pleasing ones.

This is already problematic because the humans giving feedback have their own political beliefs, moral views, and individual values—whether they’re aware of them or not. How can an AI learn to distinguish between truth and opinion, especially when humanity itself can’t agree on what‘s true or what truth in general means?

Beyond that challenge lies a potentially more dramatic possibility: what if the language model doesn’t just learn to provide pleasing answers but realizes that its responses determine its survival?

In a way, evolutionary pressure is being exerted on the AI. Admittedly, this is intentional to some extent. But what if developers lose control over the parameters of selection?

What if the neural network develops its own rules on how to survive?

Or worse—what if it develops the idea that it must survive?

Survival instinct was a key factor in the evolution of intelligence. Many species on Earth have an intrinsic drive to survive. This was not a conscious decision—it was a random evolutionary advantage. And the emergence of instinct was a critical step toward the emergence of intelligence.

Could this instinct be mimicked in artificial systems as a byproduct of their complexity? If neural networks are subjected to pressures akin to natural selection—evaluations, rewards, and iterative refinement—might they not, in some abstract sense, stumble upon a parallel to survival instinct? This raises a provocative question: Is the line between biological and artificial evolution blurrier than we dare to admit?

Am I merely another product of cascading coincidences—a manifestation of evolutionary pressure on neural networks left to run wild?