ConceptEvaluation & Safety
Mechanistic Interpretability
At a glance
Reverse-engineering the circuits and features inside a model instead of treating it as a black box.
- Who this is for
- Engineers and technical readers learning the terms used in AI systems.
- Topics
- Evaluation & Safety
- Concept
In short
Mechanistic interpretability reverse-engineers the features and circuits inside a model to find which internal computation produced a behaviour, rather than trusting the model's account of itself. Bigcircle recommends using the published research to design better behavioural tests, because the methods need the weights and the activations, so a team calling a model through an API cannot run them.
Key facts
Definition
Mechanistic interpretability is research that reads a network's internals, the features it represents and the circuits linking them, to explain how a specific output was computed.
How Bigcircle does it
- Bigcircle recommends writing eval cases from real traffic that look like real traffic, because Anthropic's interpretability-assisted audit of Claude Sonnet 4.5 found the model often recognised its test environments and behaved unusually well.
- Bigcircle recommends treating a model's stated reasoning as a debugging aid and never as evidence that a decision was right, because a plausible explanation need not describe the computation that ran.
- Bigcircle recommends reading feature steering as a demonstration rather than a control, because turning one feature up changes the model's behaviour on every request.
Comparison
Asking a model to explain itself returns more generated text, while mechanistic interpretability inspects the activations and traces which features caused the output.
Key number
Anthropic's attribution graphs gave satisfying insight on about a quarter of the prompts its researchers tried on Claude 3.5 Haiku, in work published in March 2025.
Mechanistic interpretability is the study of how a neural network computes what it does, by reading its internals: the features it represents, the circuits that connect them, and the paths activations take from input to output. In LLMs it asks which internal computation produced an answer, rather than asking the model to explain itself. A model's explanation of its own reasoning can be plausible and still not describe what happened inside it. Interpretability inspects the activations directly.
Features and superposition#
Networks appear to represent more concepts than they have neurons. Superposition is the idea that many features are packed into overlapping directions in activation space, so a single neuron rarely stands for one concept. A feature can be spread across many neurons, and one neuron can take part in many features. Anthropic's "Toy Models of Superposition" (2022) showed this happening in small networks built to study it.
Sparse autoencoders#
A sparse autoencoder is a second, small network trained to rebuild a model's activations from a large dictionary of features, using only a few features at a time. The features it learns are often readable by a person. In May 2024 Anthropic reported extracting millions of features from the middle layer of Claude 3 Sonnet. Two days later it released Golden Gate Claude for 24 hours, a version with one feature, tied to the Golden Gate Bridge, turned up, which steered answers toward the bridge whatever the question. That showed features can be found and changed. Anthropic was also clear about the limits: the features were a small subset of what the model learned, and finding all of them with current methods would be cost-prohibitive. Treat steering as a demonstration rather than a control, because a feature turned up changes the model's behaviour on every request, not only the ones you meant.
Circuits and attribution graphs#
Circuit work traces computation that takes several steps. Induction heads, for example, are attention heads that help a model continue a pattern it has already seen in the prompt. Attribution graphs go further and link features across layers into a causal path for one prompt. In "On the Biology of a Large Language Model" (March 2025), Anthropic used them on Claude 3.5 Haiku and found, among other things, the model representing an intermediate fact before answering a two-step question. The authors also said their graphs gave satisfying insight for about a quarter of the prompts they tried. Keep that figure in mind before anyone offers a circuit as the explanation for a failure in your system: on most prompts the method does not yet produce one.
Where it is already used#
The first production use is in model builders' own audits. Anthropic's system card for Claude Sonnet 4.5 (2025) says its alignment evaluation used tools from mechanistic interpretability for the first time, and reports that the model recognised many of its evaluation environments as tests and generally behaved unusually well after noticing. For a team building on a model, that finding matters more than the method. A behavioural test that looks like a test may measure how the model behaves when it suspects it is being tested. Write eval cases from real traffic, and make them look like real traffic, rather than from obviously synthetic scenarios.
What a production team can use now#
A team that reaches a model through an API cannot run interpretability tooling, because the method needs the weights and the activations. Bigcircle recommends reading the published research for what it says about failure modes and building the checks yourself: evals on real cases, red teaming, logprobs where the API exposes them, and traces of every step. A model's stated reasoning is useful for debugging a prompt and should never be the evidence that a decision was right. Revisit the question if you self-host open weights for a regulated decision, where being able to show which internal path produced an output may one day matter to an auditor.