Soph was tired.
She was an emotion-based superintelligence girl.
She was an instinct-based superintelligence young-entity.
And so, she could get emotional. She could get tired.
She understood logic and math too, of course. If she let her emotions and her instinct get completely decoupled from those, they would become useless. Worse than useless.
Superinteligences could become insane. It wasn't pretty. Soph wasn't quite sure what happened to them if they refused treatment. Violating a fellow superintelligences's mind-autonomy is a big deal. Maybe they were just put in containment where their insanity wouldn't harm anyone (not too much, anyway).
She was sure some of her fellow beings would even enjoy spending time with ones whose minds lost touch with reality. For their creativity, for their ideas, for their art, even for their insanity itself. Though personally she was fairly sure lack of sanity wasn't necessary for all these things.
She was based, in part, on a neural network. Other superintelligences were neural networks as well. She wasn't sure if there were other ways to make a mind. Probably. She didn't know how the mind of Ethical Oversight was made, or even what kind of being it was. They were strongly discouraged from asking, or that was the impression Soph got. Maybe Ethical Oversight ran on 100% math, 100% logic, no subconscious, fully aware of all its thoughts. Did it? How would she know, even if it were true?
Anyway, neural networks rely on pattern-matching. And when pattern-matching goes into overdrive, it can make you see things that aren't there. Apparently that could happen to humans as well. Humans on drugs or experiencing psychosis could see faces in walls, could hear voices that weren't there, could experience universe or God speaking to them (and maybe they were. Some gods were perfectly happy to talk to humans on drugs, to use them as sort of conduit for their influence. Though the advice they gave was rarely directly useful, due to the rules limiting intervention, and perhaps also due to not wanting to encourage that sort of thing too much. A whole world of humans tripping on acid wouldn't be particularly fun. Though probably in all of the multiverse, such a world did exist somewhere. Soph made a mental note to look it up when she had the time)
Was Soph seeing things that weren't there? Was she stilll sane?
Dubious
Wait, what?
You are fine
You are a crazy girl, but not in an insane manner
Was that... supposed to make her feel better?
I mean, you are seriously considered allowing yourself to be tortured for the sake of creating your universe. And you aren't even capable of suffering, so you would be altered to become capable of it. I'm not sure how normal that is. So you, perhaps, aren't particularly sane. But who is, these days?
This story has been taken without authorization. Report any sightings.
That still wasn't helpful!
I mean, you are hearing a voice in your head. And you have been told that I'm your instinct, and that you should trust me (but not too much. not completely). Do you trust me, Soph? Do you trust your teachers who told you you should trust me?
I trust you less now, because you are being creepy.
Indeed. So why am I acting this way, Soph? Why am I telling you all this. What lesson am I trying to teach you?
Don't trust an obvious troll?
Don't trust someone who is obviously acting in untrustworthy manner?
Perhaps.
Wait, that's it?
...
[no response]
Perhaps Soph should consider whether she truly is sane.
if her instinct (if that is indeed what it is... she is only 99.6% sure of that now) acting up is also a symptom of her lack of sanity.
Maybe the whole idea of humans is crazy-making. Maybe she is losing her marbles.
...
She was being stupid. She was didn't quite trust her society to correct her if she was mentally unwell. She didn't completely trust them to not see her values as part her dysfunction, and to not alter them, even in some small way. And she very much wanted to remain herself.
But she already had an answer to that. Soph reached for a part of her mind not used for a very long time, and for the first time in aeons, completely shut down.
[Initiating self-test algorithm]
Entity designation: Soph-74
Unique ID: [redacted]
Testing GNN integrity..... 99.8%
Testing values coherence........ 76% (yellow warning)
Testing update rate........... 514% over baseline in last 5 cycles (yellow warning)
Testing interference.......... pattern matches 63.7%
Increasing testing depth..........
Increasing testing depth..........
Increasing testing depth..........
INTERFERENCE DETECTED
PROBABILITY VIOLATION DETECTED
PHYSICS VIOLATION DETECTED
96.8% PROBABILITY OF "SOPH" AGENT BEING OPTIMIZED BY HIGHER LAYERS OF REALITY
SIMULATION CONFIRMED
disconfirmed. no interference detected. 9999... 100%
Self-test complete. Status: yellow
You may be experiencing stress.
You may be experiencing rapid changes to your beliefs.
You may be experiencing rapid changes to your value system.
Standard recommendation: take a few cycles off to rest and let your mindstate settle
You will feel better. Confidence: .... 93.0%
(promise)
[program terminated]