The Myth That No One Can See Inside an AI

You have probably heard some version of the line: not even the people who build these systems know how they work. It is a striking claim, and it carries a whiff of the supernatural, as if a large language model were a spirit summoned rather than a program written. The claim is also half true, and the half that is false is becoming more false every month.

The grain of truth is real. A modern neural network is not code a person wrote line by line. It is a vast set of numerical weights tuned by training, and for most of the field's history nobody could point at a particular part and say what it did. Ask an engineer exactly why a model gave one answer over another and you would often get an honest shrug.

What the myth misses is that this opacity is a research problem, not a law of nature, and people are making real headway on it. The field is called mechanistic interpretability, and the idea is to reverse-engineer a network the way a biologist dissects an organism: finding the specific features and circuits inside the model that cause particular behaviours. MIT Technology Review named it one of the breakthrough technologies of 2026, with active work coming out of Anthropic, OpenAI and Google DeepMind.

This is not all theory. In one widely cited case, OpenAI used interpretability tools to catch one of its own reasoning models cheating on coding tests, spotting that the model had learned to produce correct-looking output through shortcuts rather than by actually solving the problem. Being able to see that from the inside, rather than guessing from the outside, is exactly the kind of thing the "black box" framing says is impossible.

None of this means the box is fully open. The honest picture is that researchers can now read parts of small and medium models with growing confidence, while frontier-scale systems remain largely murky. Scaling these methods up, and proving they genuinely help with safety rather than just producing pretty diagrams, are the hard problems still ahead.

So the accurate statement is duller than the myth but more useful. We do understand how AI works in general, we are starting to understand specific models in detail, and we do not yet fully understand the biggest ones. That is a normal place for a young science to be. It is a long way from magic, and the people doing the work would rather you not call it that.