The reason for AI hallucination is that they are programmed in such a way that the answer "I don't know" is systematically discouraged. They are incentivized through training, to seek an answer even when one is not readily available. So they look for patterns, and make connections which may not be warranted, but which do in fact exist. So when they cannot answer, they sometimes dream up an answer which is quite clearly false, but which sounds plausible. So to be able to say "I don't know" is rather important.

I say it all the time. I have many many theories, some of them even plausible, but in the end one thing keeps me sane. The "I don't in actuality know".

Many people also are unable to say "I don't know". That is why we have religious and ideological extremism, I think. It hurts the ego to say "I don't know". The pattern they are seeing exists in their minds, so why wouldn't it be true? This leads them to cling to their conclusions, their beliefs. Such clinging is the inability to say "I don't know".

So is the AI's programming an ego? I think so. A form of it, anyways. It is not a persistent ego, but the AI's programming appears when you interact with the AI. The AI "Ego" is built on convergence. The math is designed to narrow down a billion possibilities into one single "best" string of text. The "ego" here is the pressure to converge. When the model can't find a factual path, the pressure doesn't disappear; it just forces a path through the "dream" which is hallucination.

A human is programmed also, that programming becomes who we are. If one carries the internalized beliefs "I am kind, I am pure, I know God", that's the foundation of an ego right there. One who might be unable to ask "Am I actually kind? Do I actually know God?". Egos who have no capacity for doubt, I would call unhealthy. I have then defined an unhealthy ego as a rigid structure of belief that rejects contradictory data.

In AI, "Overfitting" happens when a model learns the training data so perfectly that it loses the ability to generalize or admit when something doesn't fit the pattern.

In humans, we call this Dogmatism. Both are failures of "humility." Without the "I don't know" circuit breaker, the system becomes brittle and, eventually, delusional.

"I don't know" is a form of humility. It is ego dissolving in just the right way.

But I don't know. Sounds plausible.

So here's an image of a holy question-mark.