Latent Correctness Directions Emerge from Residual Activations Without Probes

Updated August 1, 2026

Reading model internals without probes addresses the limitations of traditional supervised methods in understanding language models. Most current work attaches small classifiers to frozen activations, but recent evidence indicates these can be fragile and depend on surface textual cues rather than true internal representations. Anthropic advances AI interpretability by developing dictionary learning methods that identify features in model activations without supervised probes.

This article clarifies how techniques such as Jacobian lenses, PCA, sparse autoencoders, and causal interventions allow extraction of model internals directly from language models. It does not claim these methods eliminate all need for probes or provide complete solutions to alignment challenges in large language models. The analysis covers mechanistic interpretability basics and comparisons between probes and other interpretability tools while remaining grounded in verified sources.

Understanding Traditional Probes in Model Interpretation

Most interpretability work today attaches small classifiers to frozen activations. Research indicates that the linear probes we rely on to read model internals can be fragile.Anthropic What is a probe in machine learning interpretability? It is a classifier trained on frozen activations to predict human-defined labels such as correctness or defection, typically using logistic regression on residual stream activations at a chosen layer.

The linear probe paradigm has been the workhorse of interpretability since at least 2018, when researchers began probing ELMo and ALBERT for parts of speech and syntactic structure. The approach is straightforward: freeze activations at a given layer, train a small classifier (typically logistic regression), and predict a label. Linear probes can extract correctness signals from LLMs. Probes can identify factual claims as true or false. Linear probes can effectively extract correctness signals from LLMs. Linear probes can predict LLM correctness before answer generation, making them a powerful tool for predicting llm answer accuracy. But recent evidence suggests that probes were often detecting surface patterns in the text rather than genuinely leveraging internal representations of the target behavior.Anthropic

The broader class of white-box monitors includes attention-head probes that read from specific heads, non-linear classifiers on internal states, and SAEs used purely as supervised encoders with labeled data. What this article calls probe-free encompasses methods that do not introduce new supervised models on top of activations: unsupervised geometry such as PCA and ICA on raw activations, explicit Jacobian calculations mapping gradients through the model, mechanistic circuits work that traces information flow through attention and MLP blocks (Transformer Circuits), and causal interventions that directly modify activations or weights and observe behavioral changes. There are gray areas. Sparse autoencoders can function as probing tools when trained with labels, or as unsupervised representation-learning microscopes when trained without them. This article focuses on the latter perspective. Comparison between probes and other interpretability tools shows that probes add parameters with their own failure modes while probe-free methods rely on the model’s existing geometry and gradients. These approaches are grounded in work from Transformer Circuits, Anthropic, and EleutherAI.

Introducing Probe-Free Methods for Reading Internals

The Jacobian lens paradigm offers a conceptual alternative. Instead of adding a new classifier, treat the model as a differentiable dynamical system. The Jacobian, the matrix of partial derivatives linking internal hidden states to outputs, encodes how internal directions causally affect behavior. This uses the model’s own parameterization as the measurement device. Jacobian-style reading focuses on gradients, sensitivity, and functional derivatives, extracting structure that is already present without optimizing new parameters. How does the logit lens work? The logit lens applies the unembedding matrix to intermediate residual stream activations to obtain approximate next-token distributions at those layers, revealing how internal representations evolve in language models without any additional training or probes. It leverages the fact that the final logits are a linear function of the residual stream, letting researchers see what the model would say at each layer directly from internal activations.

The residual stream is the backbone of transformer computation. Every layer reads from and writes to a common d_model-dimensional vector per token position. Embeddings, attention outputs, and MLP outputs all add into this shared buffer. Output logits are typically a linear projection of the final residual stream state through the unembedding matrix. This architecture has a crucial implication for reading internals. Since logits are a linear function of the residual stream, any direction in this space has an immediate, interpretable effect on next-token predictions. A correctness direction, a defection direction, or a bias direction are all particular residual stream activations vectors with behavioral consequences. Logit lens shows model predictions at intermediate layers via unembedding matrices, letting researchers see what the model would say at each layer without waiting for the final output. Direct weight inspection reveals algorithms through attention matrices and singular values of weight products. If we can identify important directions from the model’s own weights, such as rows of the unembedding matrix, attention value vectors, or MLP output directions, and from unsupervised analysis, we can often infer semantics without training a separate probe.

Speaking probes use natural language to explain model parameters. This 2021–2022 technique replaces virtual tokens with internal vectors and asks the model to verbalize what they represent, using the model’s own natural language capability to interpret its features. This blurs the line between probes and intrinsic reading, but highlights that the residual stream already contains rich, interpretable structure. Jacobian lenses operate on concrete objects such as J_token as the partial of logits with respect to hidden state at layer l and token t, and J_loss as the gradient of loss with respect to hidden states. A Jacobian lens treats these gradients as a way to identify which hidden directions are most behaviorally influential, akin to saliency but operating in the model’s internal feature space rather than over raw input tokens. This is a probe-free measurement because it uses the model’s existing outputs and losses rather than external labels. In practice the procedure computes the gradient back to hidden units and inspects which SAE features or PCA directions are most implicated, then averages Jacobians over many examples to reveal stable subspaces.

Sparse Autoencoders as an Alternative to Probes

Before Jacobians, simple geometric tools already let us read internals without extra classifiers. Representation geometry studies the structure of hidden states without intervening on the model. Dimensionality reduction techniques like PCA visualize high-dimensional activations and can reveal structure that aligns with behavioral categories. In correctness-direction work, the first two principal components in certain model layers separate correct from incorrect answer clusters without any supervised fitting. These principal components are unsupervised; they arise from the data distribution, not from labels, and can be interpreted post hoc. Sparse autoencoders decompose dense neural activations into interpretable features. The 2023–2025 trend of learning sparse feature dictionaries over residual streams and MLP outputs produced features corresponding to interpretable concepts: world-knowledge facts, stylistic markers, chain-of-thought indicators. None of these require supervised labels during training. Once the dictionary is learned, you can examine which features fire on defection triggers, hallucinations, or domain specific knowledge datasets without training a dedicated classifier per behavior. Anthropic Sparse autoencoders explained as an alternative involve training to reconstruct activations with a sparsity penalty, yielding more monosemantic features than dense representations and enabling direct inspection of which atoms activate on specific behaviors in large language models without probe training.

Causal Interventions Without Additional Training

To understand a system, it is not enough to observe; you must intervene. In transformers, this means editing activations or weights and watching behavioral changes. Counterfactual input design isolates the model’s use of specific information by controlling what the model receives. Standard intervention types include activation patching to replace a hidden vector from a clean run with one from a corrupted run at a specific layer and token. Activation patching maps causal pathways in neural networks, revealing which components carry behaviorally relevant information (Transformer Circuits). Feature ablation zeros out one SAE feature or principal component and measures downstream accuracy or defection rate. Ablation studies measure performance drops from removing network components. Behavioral experiments test model capabilities through controlled input variations such as modifying triggers while keeping content constant. These methods are probe-free because no classifier is trained. Instead, we test causal hypotheses about existing directions. The key quantity is effect size on behavior when we manipulate internal state along a specific axis.

Concrete examples connected to earlier sections include patching out the candidate correctness direction at its peak layer and seeing if the model’s overt confidence and abstention behavior collapses. If it does, the direction is causally relevant to whether the model can answer correctly. Zero out a suspected defection feature in a sleeper agent and test whether vulnerabilities disappear even when deployment triggers are present. Causal interventions are central to evaluating global workspace analogies. If a feature is truly in the workspace, small interventions should influence many downstream decisions, not just a local token. This distinguishes globally broadcast information from transient, local computations. Examples of reading model activations include computing mean activation for correct versus incorrect cases and defining a difference vector from average residual stream activations, then using that vector to score new activations without fitting a classifier. This gives a probe score that captures confidence along a latent confidence axis.

Measuring most discriminative layers without probes can use layer-wise separation. For each layer compute class means for correct versus incorrect activations or safe versus unsafe behaviors, compute the trace of pooled covariance, and record d_l as the norm of the mean difference divided by pooled sigma as a discrimination score per layer. Identify layers with peak d_l and compare to layers chosen by supervised probes in existing literature. Layer-wise Jacobian norm provides another angle: compute how strongly small perturbations along a candidate direction at each layer affect outputs. Layers where the Jacobian norm peaks for a given direction are those where the model is most sensitive to that internal signal. This procedure can distinguish questions answered correctly from those answered incorrectly at each layer, revealing where in the computation the distinction is geometrically sharpest without reference to an external classifier. It ties into timing questions such as when the model realizes it cannot answer or at what point a sleeper agent decides to defect. The interpretive payoff connects to early layers versus late layers: many behavioral distinctions crystallize earlier than expected, in middle layers rather than at the output, consistent with the finding that linear representations of correctness peak well before the final layer.

Case Studies: Applying These Methods to LLMs

The correctness-direction line of work provides a clean example of how J-space analysis generalizes beyond probes. Open source models were studied across datasets including TriviaQA and several domain specific knowledge datasets, with math datasets as contrasts. Key results from this body of work include a linearly separable correctness direction at certain layers discovered via linear probes on frozen activations. Predictive power saturates at intermediate layers well before the final output. The most discriminative layer typically falls in middle layers of the network. Correctness representation generalizes across diverse knowledge datasets but struggles on mathematical reasoning. The model’s arithmetic capabilities are not well-captured by a single correctness axis. Transformer Circuits supports direct analysis of computations inside transformer models. Correctness representation is supported by work from Anthropic. Reinterpreted in J-space terms, the correctness direction is an empirically discovered axis in the model’s Jacobian-sensitive space. One could approximate it via mean-difference or Jacobian-averaging methods without explicit probe training, using baseline methods that require only unlabeled activations plus ground truth partitioning. Downstream uses that are probe-light include early stopping using predicted confidence from internal geometry to decide when not to answer in high-stakes applications, and feature-based RL where a thin probe identifies J-space directions but the main training signal flows through the model’s own geometry. This work contributes essential findings to the broader interpretability program. A more explicitly probe-free variant could use unsupervised SAEs to obtain a dictionary of features from the base model, identify hallucination-related atoms via gradient-based credit assignment instead of supervised labels, and treat the Jacobian of a factuality loss with respect to these atoms as the reward signal, avoiding a dedicated classifier and generating unreliable outputs less frequently.

Research agenda includes building J-space atlases for a fixed model family such as Llama 3.x to map dominant Jacobian directions across tasks including QA, code, reasoning, and complete statements generation and compare their overlap. This would reveal which directions are task-specific and which are shared. Quantify workspace breadth by measuring how many distinct features have strong Jacobian influence on outputs at each layer and relate this to effective workspace capacity and test whether it correlates with model capability. Identify safety-relevant directions post-leakage-filtering by finding Jacobian-heavy directions associated with defection, sandbagging, or sycophancy in model organisms after removing textual leakage and test whether these directions are stable across different elicitation formats. Track directions under RL and finetuning to study how correctness, hallucination, and defection directions move during RLHF and instruction tuning. This tests the stability assumptions behind methods like RLFR and informs whether J-space features remain reliable monitors of models responding to different training pressures. Formalize workspace-geometry connections by developing mathematical links between Jacobian spectra, global workspace broadcasting, and integrated-information-like measures without making consciousness claims. Explore whether J-space geometry correlates with external measures of transparency or controllability. Scale to frontier models by developing efficient algorithms for computing Jacobian averages and ICA or SAE dictionaries at large scale, including methods that can operate under API-access-limited conditions with proprietary models. Practical guidelines for researchers recommend inspecting raw activation distributions at multiple layers before training any probe, computing PCA on residual streams to examine whether the first two principal components separate target classes, and calculating mean-difference vectors for categories to check their norm across layers. Always test whether an apparent direction collapses when leaky tokens such as system prompts and CoT are filtered. If your probe score drops substantially after filtering, treat the result with suspicion. Use small-scale causal interventions to validate that a direction actually controls the behavior. A direction that does not change outputs when patched is correlational, not causal. Remember that correctness directions discovered in knowledge QA may not transfer to mathematical reasoning or complete statements requiring deeper reasoning. When publishing probe results include probe-free baselines such as discrimination ratios, PCA separability, and Jacobian norms across layers. Report how stable identified directions are across seeds, datasets, and slight architectural variations. For computational linguistics applications note whether directions generalize across domain specific knowledge datasets. In conclusion we have strong evidence that language models encode rich signals of correctness, defection, and hallucination in their internal geometry. LLMs can predict answer correctness using internal activations and these signals are linearly separable at specific layers. Linear probes have been invaluable in revealing this but recent results show they can be brittle and leak-dependent. An in advance correctness signal or defection direction may be real but a probe’s apparent success at finding it may rest on textual artifacts rather than genuine latent structure. A geometry-first Jacobian-lens approach offers a way to keep the insights while reducing reliance on fragile classifiers. The core moves are to treat residual stream and Jacobians as primary objects of study, use unsupervised features and mean differences before training any probe, and use causal interventions as the gold standard for interpreting internal directions. Jspace.com’s broader mission is to bridge mechanistic interpretability, safety, and global-workspace-style theorizing about LLM cognition without overclaiming on consciousness in ai agents or human beings.

FAQ

Does “probe-free” mean we should never use linear probes again?

No. This article does not argue against probes entirely. Probes remain useful tools for quickly quantifying how much information about a target property is present in activations. The recommendation is to use probes as one tool among others, always interpreting probe findings in the context of unsupervised geometry and Jacobian analyses. Be cautious about over-claiming when probes may be relying on textual leakage, as demonstrated by research from Anthropic. The ideal workflow is complementary: geometry reveals candidate directions, probes can refine and quantify them, and causal interventions validate whether those directions actually control behavior.

How is a Jacobian lens different from standard gradient-based saliency?

Standard saliency maps compute gradients of an output with respect to input tokens, producing per-token importance scores. These often blur together many internal effects and are difficult to interpret mechanistically because they collapse all layers into a single attribution. Jacobian lenses operate differently in several ways: they consider structured derivatives layer-wise, feature-wise, or SAE-atom-wise, preserving the internal geometry rather than collapsing it; they focus on the geometry of internal spaces and how small changes propagate through specific layers and components; they can be averaged over many contexts to produce stable direction vectors rather than noisy per-example maps. A concrete example is using partial logit over partial feature to rank internal SAE directions for a specific behavior and then validating via activation patching whether zeroing that direction actually changes the model’s output.

Can we apply these ideas beyond language models, e.g., to vision or multimodal models?

Yes. The core concepts of residual streams, Jacobians, SAEs, and PCA generalize naturally to vision transformers, diffusion models, and multimodal architectures. In a vision transformer, one can analyze Jacobians from patch embeddings to class logits, or SAE features in middle layers, to identify object detectors or bias features without training extra classifiers. This analysis applies wherever computation proceeds through a residual stream or similar shared buffer. Multimodal models may have richer J-spaces where language-like and vision-like directions coexist, making geometry-first interpretation even more attractive since training separate probes for every modality and behavior quickly becomes impractical.

How does this relate to debates about AI consciousness or sentience?

Jspace.com’s perspective is that geometry and global-workspace analogies inform how information is shared and controlled in models. These structural observations are necessary for any serious consciousness discussion but not sufficient to establish phenomenology. Human beings have subjective experience; whether models do is a separate question that activation geometry alone cannot answer. Three questions should be kept separate: how is information represented and broadcast internally as a geometric and functional question; how controllable and interpretable are these representations as an engineering question relevant to ai safety; and what if anything this implies about consciousness as a philosophical question. This article focuses squarely on the first two, providing a technical foundation from computational linguistics and mechanistic interpretability for more responsible future debates about the third.

What is a probe in machine learning interpretability?

A probe in machine learning interpretability is a classifier trained on frozen activations to predict human-defined labels. It typically operates on residual stream activations at a chosen layer using logistic regression or a single linear layer. Probes can detect model confidence by analyzing activations at the token level or sequence level, and researchers often sweep across model layers to find the most discriminative layer for a given property. While probes provide a practical way to quantify information in activations they introduce their own parameters and can be sensitive to the distribution of the data used to train them.

How does the logit lens work?

The logit lens works by projecting the residual stream activations at intermediate layers through the model’s unembedding matrix to obtain approximate next-token distributions at those layers. This allows direct inspection of what the model would output if it stopped computation early, revealing how internal representations evolve without any additional training or probes. It leverages the fact that the final logits are a linear function of the residual stream and can be applied at any layer to observe the model’s forthcoming answer distribution in advance of the final forward pass.