What the J Lens Protocol Isolates That a Logit Lens Cannot

Fact-checked by the J-Space editorial team

J Lens Protocol for Mechanistic Interpretability Experiments

Updated August 15, 2026

Mechanistic interpretability research needs a named lab procedure that joins residual-stream readouts to first-order causal tests, because post-hoc saliency and feature browsing do not isolate mechanisms in large language models. The NIST AI Risk Management Framework treats explainable and interpretable as one of the core characteristics of trustworthy AI and is the main U.S. government hook for demanding more than post-hoc heatmaps.

This article specifies that procedure: pre-registration, residual-stream instrumentation, lens measurement, Jacobian-guided patching, negative controls, and reporting limits. It does not claim that a local circuit on a synthetic task is a safety property of a production system, and it stays neutral on consciousness debates. J-Space analysis constrains causal structure. It does not establish phenomenology.

What Mechanistic Interpretability Research Allows a J Lens to Claim

The J Lens protocol is a joint residual-stream lens plus first-order (Jacobian / attribution) causal stack, not another synonym for the logit lens, and that distinction is the difference between a ranked sensitivity map and a mechanism. Standard mechanistic interpretability aims to reverse-engineer neural networks by identifying features, circuits, and activation patterns that explain model behavior; tools such as sparse autoencoders, the logit lens (which projects intermediate activations to vocabulary space), and activation patching have powered discoveries from induction heads to indirect object identification circuits, but those methods are primarily observational or coarse-grained causal: they tell you what is active or what breaks when removed, without always quantifying the local sensitivity landscape that links internal states to outputs. Post-hoc XAI saliency, integrated-gradients heatmaps, and related attributional methods answer a different question again: which input tokens or pixels a score depends on, not which residual-stream directions, heads, or MLP writes implement the precise computations transforming inputs into outputs. SAE feature-browser tourism is a third activity, useful for generating labels and for decomposing activations, but browsing a dictionary is not an experiment until a pre-registered behavior metric, a contrast pair, and an intervention are attached. Mechanistic interpretability research therefore needs a protocol that can sit on top of those tools without collapsing into them. Jacobian-centric analysis supplies the missing derivative layer: the Jacobian of a neural network, or of a hooked internal module, captures how small changes in one part of the network affect another, so that features remain the candidate units of representation, circuits remain the candidate subgraphs of features and weights, and the Jacobian connects those units quantitatively. The result is not a replacement for existing interpretability workflows. It is a structured overlay that uses local derivatives to prioritize, test, and refine mechanistic hypotheses generated by other methods, and it is intended for experiments on toy models and large models alike, with particular emphasis on transformer models and RL agents that develop internal world models. The protocol assumes basic familiarity with mechanistic interpretability research: if you already know what an induction head is and have used activation patching, you are the intended reader. It is not a first tutorial in neural networks, and it does not treat “AI is a black box” as a finding.

The only claims the protocol is allowed to output are localized readout (which residual, head, MLP, or feature coordinates currently decode or rank as carrying the behavior), causal necessity (which of those coordinates change the pre-registered metric under a specified intervention, relative to negative controls), and path isolation (which routes through heads, MLPs, and residual-stream directions are sufficient to carry that effect under path patching rather than full-activation replacement). Those three claims are already a high bar. They are not a license to infer that neurons represent distinct features everywhere, that the network contains complex internal world models in the everyday sense, or that a local circuit is a safety property of a deployed system. J-Space is the space of Jacobians over a model’s internal representations: for a given behavior you study local maps of the form state-out with respect to state-in for key layers, heads, or features, and unlike activation space (which tells you what is present) J-Space tells you what the local linearization says matters. The Jacobian Lens is the mapping from a chosen internal subspace, such as a sparse-autoencoder feature basis or a principal-component decomposition, to its local causal influence on a behavior of interest; it ranks intermediate coordinates by sensitivity and, when composed with the unembedding, can project those coordinates into vocabulary space. Internal world models enter as a scientific question, not a slogan: when models trained on language or control appear to track entities, occluded objects, or latent task state, J-Space is one way to measure whether those latent coordinates have first-order influence on downstream model behavior. Global-workspace-style ideas enter the same way, as operational mappings (residual streams as candidate shared workspaces, attention heads as candidate broadcast specialists) that can be tested by derivative-based broadcast profiles, not as a verdict on consciousness. A world model is any learned predictive model of environment dynamics; an internal world model is such a model embedded within a larger agent or language model, not necessarily an explicit separate module. A workspace is a subspace with high broadcast to many modules; a bottleneck is a low-dimensional compression point; they may overlap and are not identical. These concepts map to actual tensors in transformer models (residual-stream vectors, attention-head outputs, MLP activations) and in RL policies (latent state vectors, value estimates, policy logits). The protocol is designed for neural network models where you can access gradients and intermediate activations, including open-weight language models, vision transformers, and RL policies with differentiable components. Typical use-cases include interpreting specific model components in a language model (induction heads, IOI-style binding circuits), mechanistic interpretability in world models for reinforcement learning, and identifying high-broadcast residual directions in large models. The protocol is not aimed at non-differentiable search or retrieval modules without gradient paths, closed-source APIs without gradient access, or complex or proprietary models whose internals cannot be logged at the required granularity. Toy models remain the correct starting point: a toy model architecture with known or constructed ground-truth circuits lets you validate J Lens findings before moving to frontier-scale large models where computational costs rise by orders of magnitude. Policy language is a receiver, not a substitute for methods: NIST already lists interpretability among the characteristics of trustworthy AI, and the OECD AI Principle 1.3 on transparency and explainability is the international counterpart to a reproducible mechanistic protocol. Neither document certifies a circuit diagram.

Mechanistic interpretability versus post-hoc XAI

• Mechanistic interpretability asks which model components implement a behavior and whether those components are causally necessary on a specified path.
• Post-hoc XAI asks which inputs a score is sensitive to, usually after the fact, and typically without a residual-stream circuit or a null-circuit control.
• Saliency, SmoothGrad, and integrated gradients can show input-feature influence. They do not localize heads, MLPs, or residual-stream directions.
• A heatmap that highlights a name token on an IOI prompt is compatible with many internal algorithms. A J Lens writeup is not done until a contrast pair, a lens curve, and a failed or successful intervention are reported together.
• NISTIR 8312 is explicit that explanation, meaningfulness, explanation accuracy, and knowledge limits are separate validity conditions. A pretty attribution plot can fail all four. NISTIR 8312

J-Space, circuits, and internal world models

• J-Space: Jacobians over hooked internal representations for a pre-registered metric; the silent workspace of local causal weight, not a catalog of whatever is active.
• Jacobian Lens: a ranked map from a chosen basis (residual coordinates, SAE features, head outputs, hand-crafted directions) to that metric, optionally decoded through the unembedding.
• Circuits: subgraphs of heads, MLPs, and residual-stream directions. High Jacobian coordinates are candidate nodes, not finished circuits.
• Internal world models: latent coordinates that track environment or entity state and that can be tested by dynamics Jacobians and targeted perturbations.
• Workspace-like broadcast: a feature or direction with high Jacobian norm across many downstream modules is a candidate hub. Candidate status is not phenomenology.
• Linear representation features and linear feature representations are working hypotheses about directions in activation space. They are inputs to the protocol, not conclusions of it.

Scope: models, questions, and non-goals

• In scope: differentiable neural networks with hookable activations and gradients; open-weight decoder-only transformer models; small actor-critic or latent-dynamics agents.
• In scope as questions: induction, indirect object identification, factual recall, simple refusal or syntax tasks, object permanence in a grid-world, local latent dynamics in an RL world model.
• Out of scope: API-only access; discrete search without a gradient path; claiming that interpretability research has audited a production model because a GPT-2-class circuit exists.
• Out of scope as rhetoric: alignment eschatology, SAE tourism with no experimental design, and listicles of papers with no protocol the reader can run.
• Starting scale: 2 to 4 layer transformers, small CNNs, simple actor-critic nets. These are environments where you can compute denser Jacobians and where ground-truth circuits are simpler.

Residual-Stream Instrumentation in a Language Model

Every J Lens measurement is a map from residual activations to a behavior metric, and in a decoder-only language model that metric is almost always a function of vocabulary logits, so the unembedding is not an optional visualization trick: it is the readout geometry against which every layer-wise lens is defined. The residual-stream / virtual-weight decomposition published in the Transformer Circuits framework is the de facto instrumentation standard that logit-lens, tuned-lens, and activation-patching experiments all inherit: each block writes an additive update into a shared residual stream, attention heads and MLPs are the writers, and later layers can be read, to first order, as linear maps on that stream. The logit lens assumes that intermediate residual vectors are already close enough to the unembedding basis that W_U r_L is a meaningful vocabulary distribution; the tuned lens relaxes that assumption by learning an affine translator per layer so that early-layer geometry is not forced to speak the final basis; decoder or future-lens variants ask what a later block would decode if it were given the current residual. None of those probes is causal. A J Lens stack uses them as the dependent variable of the measurement pass and then asks the first-order question: which hooked coordinates change the pre-registered logit difference, KL, or exact-match score. The minimum viable stack is an open-weight decoder-only model, cache hooks at named residual / head / MLP points, a fixed tokenizer, and frozen weights. TransformerLens-style named hooks, nnsight-style intervention contexts, or an equivalent register_forward_hook discipline are acceptable; mlxterp is acceptable for local Apple Silicon prototypes. Conceptual prerequisites are features, circuits, activation patching, the residual stream, backpropagation, Jacobian-vector products, and the difference between correlation in activation space and causal effect via intervention. Technical prerequisites are Python and PyTorch (or JAX or MLX), experience hooking model internals, and enough GPU or Apple Silicon memory to retain intermediate tensors with gradient access. Half-precision inference is fine. Float16 backward passes are not a default for ranking features, because precision noise can reorder a sensitivity list. Closed APIs are out of scope: without activations and gradients you can still do careful behavioral interpretability, but you cannot run a J Lens causal stack, and you should not describe API prompting as mechanistic interpretability.

What to hook, name, and refuse to invent

• Residual stream at each layer, pre and post attention, pre and post MLP.
• Attention-head outputs and attention patterns (the value-weighted writes, not only softmax maps).
• MLP activations and, when a dictionary exists, SAE feature activations on the same residual sites.
• Stable names such as blocks.6.hook_resid_pre and blocks.6.attn.hook_result, documented in the experiment config so another lab can rerun the cache.
• Retain gradients on cached tensors. A detached cache is an observational log, not J-Space access.
• Do not log “everything” on a 70B forward pass and call the resulting dump a protocol. Targeted slices are part of the design, not a compromise of it.

Pre-Registration, Contrast Pairs, and Toy Models

J Lens starts from a falsifiable question about model behavior, written down before any activation is cached, because a Jacobian landscape without a null circuit is an invitation to harvest whatever lights up. For each experiment, define the input distribution, the behavior metric, the expected layer / head / MLP or residual-direction locus, the null circuit (what should not matter), and the failure cases in which the hypothesized mechanism should be absent or reversed. Vague questions such as “what does layer 6 do?” produce ranked noise. Precise questions produce stop/go tests. Concrete templates already exist in interpretability research and should be reused rather than rebranded: indirect object identification on IOI-template sentences with logit difference between the correct indirect object and the distractor; object permanence in a small RL agent on trajectories with occlusion events, with reward-to-go or retrieval success as the metric; chain-of-thought versus direct prompts for multi-step arithmetic, with correct-answer log-probability as the metric and with failure cases where CoT degrades performance. Training models on synthetic tasks is not a demotion. It is how you control the data distribution so that you can say what circuits should emerge, and it is why toy models remain the correct first instrument for mechanistic interpretability research. Begin with 1-4 layer transformers with 1-8 heads, trained on synthetic tasks such as IOI, modular arithmetic, or repeated-token induction, where ground-truth circuits are simpler and full or nearly full Jacobians are still memory-feasible. Mid-scale open language models in the 1-7B range are the next tier, and at that scale you restrict Jacobians to targeted slices: specific features, token positions, or layers. For reinforcement learning, prefer environments in which latent state is at least partly human-interpretable (MiniGrid-style partial observability, memory-requiring procedural tasks, differentiable latent dynamics) so that a dynamics Jacobian can be checked against a world variable rather than against a vibes-based story about agents that develop causal models.

Construct clean versus corrupted prompts the way IOI-style patching requires, and decide stop/go metrics before scanning every head. A clean prompt is the canonical task instance on which the language model produces the target completion; a corrupted prompt is a minimal edit (swapped names, ablated cue, scrambled label, off-distribution entity) that should break the hypothesized computation while leaving surface statistics similar. That pair is the experimental unit for both the lens curve and the later patch. Stop/go metrics must be numeric and pre-declared: logit difference, KL to the clean output distribution, exact-match rate, policy success, or reward-to-go. Scanning 144 heads after seeing the heatmaps is not a confirmatory analysis, and interpretability research that treats the scan as the finding is doing exploratory visualization under a causal name. Data design is part of pre-registration because J-Space is local: a Jacobian is a statement about one operating point. For a language model, curated minimal pairs, hand-constructed reasoning chains with matched skip-step controls, and synthetic factual items with controlled frequency are the usual families. For RL, annotated trajectories and environment variants that isolate memory, planning, or spatial reasoning are the usual families. Keep initial Jacobian-heavy sets modest (on the order of 10^2 to 10^3 examples). Each example may require one or more backward passes. Small, versioned, documented sets beat large undocumented scrapes. Features probing involves training probes or dictionaries on the same residual sites you intend to differentiate; those probes are auxiliary instruments, and their accuracy is not a causal score. If you cannot write the null circuit and the stop/go rule in advance, you are not ready to compute a Jacobian.

Pre-registration checklist

• Task name, input distribution, and exclusion rules for malformed prompts or trajectories.
• Scalar or low-dimensional behavior metric, including the exact token positions or reward horizon.
• Hypothesized locus (layers, heads, MLPs, residual directions, SAE coordinates) and an explicit null circuit.
• Clean / corrupted construction rule, including what is matched (length, frequency, entity class) and what is deliberately broken.
• Stop/go thresholds decided before the head scan: for example, a patch must recover a pre-declared fraction of the clean-versus-corrupted logit gap, and a random-head control must not.
• Negative-control families: scrambled labels, random heads at the same layer, off-distribution paraphrases, and at least one unrelated task for off-target damage.
• Multiple-comparisons plan: how many heads, layers, and residual directions will be scanned, and what correction or hold-out rule applies.
• Compute budget and scale justification: why this toy model or this open checkpoint is scientifically enough for the claim.
• Promotion rule: lab note, triangulation candidate, or eval-informing result. Default is lab note.

Model choice and why toy models are not a concession

• Toy models: full or dense Jacobians, cheaper negative controls, and a chance to catch autograd bugs against known circuits.
• Mid-scale open models: targeted slices only; still enough to test whether a mechanism survives naturalistic vocabulary and paraphrases.
• Frontier-scale large models: last stage, not first; only after a cheaper stack has produced a hypothesis that is still live.
• RL latent-dynamics agents: in scope when the latent and the reward or policy head are differentiable and loggable.
• Reproducible loading: versioned configs, seeds, tokenizer revision, and frozen weights. Later J-Space numbers depend on all four.
• The IOI literature already showed that GPT-2-small-scale circuits can be isolated with contrast pairs and activation patching. That is the causal half of any J Lens stack, and it is a reason to stay small until the claim requires otherwise.

Measurement Pass, Causal Pass, and World Models

The measurement pass treats lens readouts as the dependent variable, not as the mechanism, and it logs the full layer-wise lens curve for the hypothesized token or feature rather than a single cherry-picked heatmap. On each clean and corrupted example, cache residual streams, decode them with the chosen lens (logit, tuned, or a frozen affine translator you have justified), and record the metric trajectory across layers at the token positions named in the pre-registration. Early-layer lens readouts that “already know the answer” are usually a basis-change artifact: if the residual is not yet in the unembedding geometry, W_U r_L can assign mass to the correct token for reasons that have nothing to do with a finished computation, and a tuned lens can still leak label information if the translator was fit on a related distribution. False-positive early geometry is therefore a design input to the causal pass, not a discovery to tweet. If the lens curve peaks early and then collapses, do not “confirm” an early-site circuit with a full-activation replacement at that layer; move to path patching or to a later write that the curve actually implicates, and treat the early peak as a hypothesis about basis alignment. If the curve rises only at the hypothesized late heads, those heads become the first intervention targets. If the curve is flat, stop: the lens does not license a Jacobian fishing expedition across the whole network. Log per-example curves, not only means. A mean curve can hide a bimodal mixture in which half the prompts are solved by a different route. Combine the lens log with a first-order screen: one backward pass from the scalar metric into cached residuals, head outputs, or SAE coefficients yields a sensitivity ranking that is cheaper than a full patching sweep and that should be treated as a screen, never as standalone proof. Per-feature sensitivity J_i = d(metric)/d(a_i) is the basic object; layer-to-layer maps d(r_{L+k})/d(r_L) are the object when the question is transmission; d(z_{t+1})/d(z_t) is the object when the question is latent dynamics and spatial and temporal data flow in an RL world model. Full dense Jacobians are for toy models. Mid-scale work should use VJPs for “what does this metric depend on” and JVPs for “what does this direction do,” and should subsample tokens to the prediction site and the named cue sites.

J_i = d(metric)/d(a_i)   # J_i: local sensitivity of the pre-registered metric to coordinate i
metric = s(W_U r_L*)     # s: scalar such as logit difference; W_U: unembedding; r_L*: residual at the named site
a_i = e_i(r)             # a_i: residual coordinate, head write, or SAE feature; e_i: the chosen encoder or slice

J_i is per-coordinate sensitivity, metric is the pre-registered scalar, a_i is the hooked coordinate, and none of those symbols is a causal conclusion until the causal pass moves the same coordinate and the metric changes in the predicted direction relative to controls. Activation patching and causal tracing are the confirmation layer the lens cannot provide: they replace or interpolate activations from clean to corrupted runs (or the reverse) and ask whether the metric is restored. Path patching, unlike full-activation replacement, holds some routes fixed and so can localize a write through a head or MLP rather than blaming an entire residual vector that many circuits share. Ablation (zeroing a head write, an SAE coefficient, or a latent dimension) is a harsher test and is easy to over-interpret when superposition or redundancy is present. Attribution patching and other local-Jacobian screens are cheap and should be used to order candidates, then discarded as proof. The mapping from J-Space to manipulations is direct and should be written before the first patch: identify the top-k coordinates by sensitivity, choose a manipulation (zero, scale, swap, add a signed offset), run it on the full curated set, and measure the same metric you registered. Standard tools – activation patching, path patching, head ablation, residual steering – keep results comparable with the rest of interpretability research. Experimental controls are not optional add-ons: random-head or random-feature ablations at the same layer, low-sensitivity control coordinates that should do little, and repeated evaluation across the whole set rather than a single prompt. This is also where a misleading lens curve must change the causal design, a step that most siloed lens tutorials and siloed patching tutorials never specify. An early false-positive lens peak shifts you away from naive layer replacement and toward path isolation at later writes. A late lens peak that fails to patch is evidence against the simple readout story and a reason to look at multi-hop paths or at a bad metric, not a reason to quietly swap in a different layer. For world models, compute both d(behavior)/d(latent) and d(next_latent)/d(current_latent) at named times (for example, just after occlusion). High-influence latent directions that also persist under the dynamics Jacobian are candidates for internal causal models; they become findings only after a perturbation makes the agent behave as if the world variable changed. High-broadcast, high-Jacobian features are candidates for workspace-like representations because they influence many downstream modules at once. Candidate broadcast is still not consciousness, and J Lens does not establish phenomenology. Frame such measurements as mechanistic constraints on any later theory: whatever else the theory says, it must respect the observed causal structure.

Computing Jacobians without turning the lab into a memory experiment

• Forward pass with gradients enabled on target activations; define the scalar metric; backward or JVP/VJP into the named hooks.
• Toy model architecture below roughly 100M parameters: dense Jacobians are sometimes feasible and useful as a debugging gold standard.
• Mid-scale models trained on real text: batched VJPs; one backward pass per example per metric; read gradients at cached hooks.
• Use float32 for the backward ranking pass. Average only after per-example Jacobians are stored. J-Space is local.
• Sanity checks: d(logit_i)/d(logit_i) is 1; gradients vanish after detach; a few finite-difference comparisons on toy models catch autograd bugs.
• Log per-example Jacobian norms, top-k indices, prompt text, token positions, and metric values. Those logs are the analysis, not a leftover.

Intervention menu (confirmatory, not exploratory theater)

• Feature zeroing or scaling on SAE coordinates or residual directions implicated by J_i.
• Head ablation or counterfactual replacement of a head write from the paired corrupted (or clean) run.
• Path patching along a named route; report what the path cannot isolate as clearly as what it can.
• Latent offsets in an RL world model along a direction that is supposed to encode a world variable.
• Random-head, low-J_i, and off-task twins for every intervention that you intend to publish as causal necessity.

Controls, Superposition, and Interpretability Research Limits on Large Models

Controls that typical lens blogs skip are not etiquette. They are the difference between a sensitivity ranking and a false circuit. At minimum, run scrambled-label prompts (the surface template without the cue that should drive the metric), random-head or random-feature ablations matched for layer and tensor size, and off-distribution prompt families (paraphrases, held-out entity names, different templates, different environment seeds). If scrambled labels still produce the same lens peak, you are reading template geometry or frequency, not the hypothesized computation. If random heads at the same layer move the metric as much as your J-Space heads, you do not have localization. If the effect vanishes under paraphrase, you have a prompt-specific operating point. Off-target batteries matter for the same reason: a coordinate that implements the task and also collapses unrelated syntax or factual recall is a shared resource, and mechanistic interpretability should say so rather than silently promoting it to a dedicated module. Dataset splits belong in the same discipline. Compute the screen on one half of the modest set and reserve the other half for confirmatory patches. A mechanism that exists only in the half you stared at is a researcher degree of freedom, not a circuit. When the objects of study are neural networks with residual streams of several thousand dimensions, these controls are cheaper than another narrative figure and more informative than another un-controlled heatmap.

Multiple-comparisons cost is the other control that interpretability research routinely under-reports. A 12-layer, 12-head model already offers 144 heads; add per-layer residuals, MLP channels, and SAE dictionaries with tens of thousands of features and an uncorrected scan will always find a dramatic bar chart. Pre-declare the family you will scan, report how many tests you ran, and keep a hold-out set or a second task for confirmation. Ranking by |J_i| and then patching only the winners is a legitimate screen if you say it is a screen. It is not legitimate if the paper writes as if those winners were the only coordinates contemplated. Researcher degrees of freedom also include metric shopping (logit difference versus KL versus exact match after seeing which one moves), site shopping (resid_pre versus resid_mid versus attn_out), and basis shopping (raw residual versus PCA versus several SAE checkpoints). Any of those choices can be justified. None of them can be silently revised after the intervention table is known. The honest writeup lists the choices in the pre-registration and reports negative screens with the same status as positive ones. A plausible failed experiment is part of the protocol, not an embarrassment: high-J_i SAE features for capital-city prediction that do nothing when ablated mean the features were correlated with the computation, the dictionary missed the causal directions, or the metric was insensitive. Publish that. It is more useful to later mechanistic interpretability research than a third successful induction-head retelling.

Superposition is why single-neuron or single-direction J Lens readings are incomplete even when the ranking is stable and the patch works. Neural networks can represent more features than they have dimensions by packing concepts into overlapping directions; polysemantic units are associated with multiple unrelated concepts; and the hope that neurons represent distinct features is an empirical claim to test, not a background assumption. Sparse autoencoders are the current industrial attempt to rotate into a basis where activations are sparse and closer to monosemantic, and Anthropic’s production-model SAE program is evidence that reverse-engineering of features is now a first-class research program rather than a hobbyist add-on to toy models. SAEs complement a lens protocol. They do not replace it. A dictionary supplies candidate coordinates and human-readable labels; the protocol still has to pre-register a metric, draw a lens curve, and intervene. An SAE feature with a persuasive autointerp writeup and a near-zero causal score is a labeled tourist stop. An unlabeled residual direction that patches the metric is a better mechanistic object. J-Space can quantify overlap: when the Jacobians for behavior A and behavior B both load on the same feature, you have a numerical handle on shared use, which is one reason off-target controls are not optional. Linear representation features can still be the right working language, and linear feature representations can capture many semantic distinctions, but superposition is exactly the regime in which a single-unit story is too small. The universality hypothesis (that similar features recur across neural networks) is a transfer hope for J Lens, not a free lunch: replication on a second checkpoint is still required. Neuron visualization techniques reveal candidate semantics for high-J_i units when top-activating examples are inspected; those visualizations are labels for the intervention table, not substitutes for it. Features probing involves training classifiers or dictionaries on internal representations to test what information is accessible; accessibility is not use. Structured probes primarily test linear decodability. J Lens is for use under intervention.

What a successful J Lens result is still not allowed to claim follows from the same controls. A local circuit on a synthetic task is not a safety property of a production model; it is not evidence that the system has emergent causal world models in any sense that would settle a consciousness debate; and it is not a license to say that mechanistic interpretability has made the model trustworthy. SAE features, probes, and J Lens readouts should be triangulated rather than swapped: a probe says a variable is decodable, an SAE says a dictionary element is active, a lens says a residual decodes to a token, and only the causal pass says the coordinate was necessary on a path. The promotion rule is conservative. A protocol result can inform evals when the same coordinate fails or succeeds on a held-out distribution that the eval actually cares about, when negative controls are clean, and when off-target damage is reported. Otherwise it stays a lab note. Scaling does not relax that rule. Naive J-Space computation scales poorly with width and sequence length; full Jacobians in a 70B-class network are not a plan; and the field already knows that mechanistic interpretability requires substantial compute if it is done naively. The scalable pattern is coarse screening with cheap lenses and a few patches, VJP/JVP directional derivatives instead of dense matrices, SAE or low-rank bases instead of raw hidden states, and token subsampling at named sites. Infrastructure should be staged: consumer hardware for toy models, a single GPU for 1-7B open models, multi-GPU clusters only for pre-screened slices of large models. Budget GPU hours explicitly. J Lens is gradient-heavy compared with purely observational browsing. Integration with other paradigms should stay sequential and honest: behavioral tests find a behavior; attributional methods can narrow input cues; J Lens maps internal sensitivity; circuit analysis names the heads and MLPs. Each view misses something. Behavioral probes miss computation. Attribution misses internal pathways. Circuit diagrams miss quantitative sensitivity. J Lens misses some non-linear interactions and any mechanism that the local linearization cannot see. Dual-use and overconfidence are the remaining limits. The same tools that support enabling model alignment and mechanistic audits can also support surgical edits that bypass guardrails. Miscalibrated claims of the form “we ran a lens, therefore the model is safe” are themselves a safety problem. Present uncertainty, publish failed interventions, and do not release general attack recipes. Mechanistic interpretability can support a granular and precise understanding of some computations in some neural network models. That is the ceiling this protocol is willing to advertise.

Methodological pitfalls

• Treating Jacobian norms as causation. Large J_i is a screen. Only interventions, with controls, support causal necessity.
• Ignoring locality. A Jacobian on one template does not travel to another template without a measurement.
• Projecting toy results onto large models as if scale were only more of the same superposition and the same pathways.
• Vanishing or exploding gradients, ReLU death, softmax saturation, and layer-norm flattening that hide or invent sensitivity.
• Autograd misuse: missing retain_grad, accidental detach, ranking in float16, batch-averaged Jacobians presented as per-example facts.
• Polysemantic contamination: a unit’s influence on your metric may be a side effect of a different computation that shares the direction.

Scaling and staged compute

• Stage 1: prototype on toy models, hours, consumer hardware, dense enough Jacobians to debug the stack.
• Stage 2: validate on mid-scale open models, days, single GPU, targeted slices and VJPs.
• Stage 3: extend to frontier-scale large models only where the cheaper stages left a live hypothesis, weeks, cluster, pre-screened sites.
• Always cheaper than a blind 144-head confirmatory sweep: a pre-registered 3-5 site list informed by the lens curve.

Ethical and safety notes that do not replace methods

• Benefits that can be stated without prophecy: internal audits beyond behavior, better red-team design, a chance to notice deceptive or occluded-state circuitry before deployment.
• Harms that can be stated without prophecy: more efficient model editing, more precise adversarial prompts, false confidence in partial diagrams.
• Governance: no general attack recipes, responsible disclosure for concerning internal structure, staged release of high-risk tooling, and no claim that a protocol run is a safety certificate.
• Consciousness language stays off the causal score sheet. Broadcast profiles constrain theories. They do not settle them.

Worked Templates for Internal World Models and Overclaim Rules

A concrete GPT-2-small-scale walk-through is the right first confirmatory exercise, both because the scientific case for staying on GPT-2-class models is strong and because the compute budget is honest: hours on one GPU, not a cluster narrative. Use a 2-4 layer GPT-2-style transformer, or GPT-2 small itself when you want a public checkpoint, on synthetic repeated-token sequences or on a public IOI-style split. Define the metric as the logit advantage of the correct continuation (or the IOI name) at the named position. Pre-register the expected late-site induction or name-mover heads, the null set of early heads that should not restore the gap under path patching, and the stop/go rule (for example, ablating the implicated heads must remove a pre-declared fraction of the logit gap, and a same-layer random-head set must not). Run the measurement pass: full lens curves on clean and corrupted prompts, plus a VJP screen into every head write. Do not stop at the first layer where the lens “knows” the answer. If an early-layer readout already ranks the correct token, treat that as a basis-change warning and keep the causal targets on the late writes that the curve and the prior literature actually implicate. Patch those writes, path-patch the residual routes that feed them, and log failed interventions with the same status as successful ones. Optionally fit or load a small SAE on the residual just before the implicated heads and recompute J_i in that basis so that the head-level circuit and the feature-level ranking can be triangulated. Expected structure, if the stack is working, is a small set of late heads with outlying sensitivity, attention patterns that match the known algorithm, and large metric drops under the pre-registered ablation. That result is a tooling validation and a lab note. It is not breaking news about induction heads, and it is not a claim about broader world models. GPU-hour honesty belongs in the same paragraph: if the question is “does our J Lens implementation recover a known GPT-2-class circuit with controls,” GPT-2 small is scientifically enough; if the question is “does this mechanism survive 7B-scale naturalistic paraphrase,” you need that scale and you still do not need a 70B dense Jacobian. Required figures for any such writeup are the lens curve, the patching heatmap, a path diagram, and the failed or control interventions. Missing any one of those four is a reporting defect, not a style choice.

The same skeleton extends to factual recall in mid-scale language models and to latent world models in RL without changing the claim hierarchy. For factual recall, take an open 1-7B checkpoint, a few hundred paraphrased fact prompts, and mid-layer residuals as the suspected consolidation band. Apply a published SAE or a linear direction set you can defend, rank features by mean J_i on the correct-answer logit, and only then patch or swap the top coordinates between facts. A transfer effect (patching a France-capital direction into a Germany prompt and watching the capital token move) is evidence for a structured factual direction on that set. It is not evidence that the language model stores a complete factual graph, and it is not evidence that models trained on web text have developed a human-like encyclopedia. Neuron visualization techniques reveal semantic clusters among high-J_i features when you inspect top activations; report those as labels. For RL, take a differentiable latent-dynamics agent on a partially observable task, define expected reward or subtask success as the metric, and compute both d(behavior)/d(z) and d(z_{t+1})/d(z_t) just after occlusion. Hypotheses should name world variables (hidden object position, layout, opponent presence) and should predict both persistence in the dynamics Jacobian and a specific behavioral change under a latent offset. Interventions that create targeted errors (the agent navigates to the wrong hidden location) or blindsight-like deficits (zeroing the putative object-position subspace) are the confirmatory layer. Robustness across seeds and environment variants is required before anyone says the agent maintains internal causal models rather than a brittle stimulus-response map. These are the settings in which it is legitimate to ask whether agents develop internal world models and whether those models are causal. They are also the settings in which overclaim is easiest: a latent that correlates with occlusion is not an emergent world simulator, and a successful perturbation is not proof that neural networks in general develop complex world representations. Copyable templates should stay boring and reusable: induction, IOI-style binding, factual recall, simple refusal or syntax, and one occluded-object RL task. Each template reuses the same pre-registration fields, the same four figures, and the same promotion rule.

Evaluation, lab practice, and future work should serve the same overclaim rule rather than a progress narrative. Ground-truth benches are algorithmic toy models, compiled transformers whose internal program is known by construction, and dictionaries with human-checked feature semantics. Accuracy means J Lens-predicted sites match those benches under ablation. Robustness means the sites recur across seeds and dataset halves. Predictive validity means the signed intervention does what the writeup said it would do, within a pre-declared tolerance. Reproducibility means open configs, named hooks, versioned tokenizers, shared J-Space summaries, and a clear split between exploratory scans and confirmatory tests. New labs should reproduce the GPT-2-class induction or IOI template before proposing a novel mechanism; wrap hooks and VJPs in a shared library; and review gradient code, because autograd bugs are silent. Cross-lab comparison needs shared residual names, shared tasks, and shared negative controls, not a new brand for the logit lens. Near-term technical work that is actually on-protocol includes better large-scale Jacobian approximation, SAE objectives that care about J-Space sparsity as well as reconstruction, and cross-framework hook APIs. Automated intervention search is in-scope only if the search is held out from the confirmatory set. Training-time J Lens, multi-modal systems, and multi-agent latent spaces are open problems, not deliverables of this writeup. Transfer of J-Space patterns across architectures, quantitative measures of broadcast or world-model complexity, and the training-time emergence of circuits are open questions. The long-term goal is not explanation for its own sake. It is enough understanding of internal causal models in systems we actually deploy that evals can be pointed at named mechanisms rather than at vibes. That goal is served by conservative promotion, not by inflating a local circuit. The working loop remains: define the behavior, measure the lens curve, rank first-order sensitivity, intervene with controls, and report failures. Mechanistic interpretability research that keeps that loop, and that refuses to swap it for SAE tourism or alignment sermonizing, is doing the job this protocol names.

Required reporting package

• Pre-registration: task, metric, locus, null circuit, stop/go rule, comparison budget.
• Figures: lens curve, patching heatmap, path diagram, failed or control interventions.
• Tables: per-site causal scores versus random-site baselines; off-target damage on at least one unrelated task.
• Compute: hardware, precision, number of backward passes, and why the chosen scale is enough for the claim.
• Promotion label: lab note (default), triangulation candidate, or eval-informing. Never “the model is safe.”

Copyable task templates

• Induction on repeated tokens: logit advantage of the completion token; late-head locus; random-head null.
• IOI-style binding: logit difference of indirect object versus distractor; name-mover / backup-head hypotheses; name-scrambled controls.
• Factual recall: correct-entity logit on paraphrases; mid-layer residual or SAE locus; cross-fact patching.
• Simple refusal or syntax: contrast legal versus illegal continuations; do not generalize to “alignment is solved.”
• Occluded-object RL: success after occlusion; latent subspace locus; dynamics-Jacobian persistence test.

What stays a lab note

• Any single-checkpoint, single-template circuit without paraphrase or seed controls.
• Any SAE-only narrative with no intervention table.
• Any early-layer lens peak presented as a finished computation.
• Any jump from a synthetic circuit to a production safety claim, including claims about enabling model alignment as an achieved result rather than a possible use.
• Any workspace or consciousness gloss that is not operationalized as a broadcast measurement with an explicit non-phenomenological caveat.

FAQ

How expensive is J Lens in terms of compute compared with standard interpretability methods?

J Lens is typically more compute-intensive than purely observational methods because it relies on gradients and sometimes partial Jacobians. For toy models and curated sets of hundreds to a few thousand examples, experiments usually fit on a single modern GPU or high-end Apple Silicon machine in hours to days. VJPs instead of dense Jacobians, plus restriction to named features or layers, keep 1-7B open models on one GPU. Frontier-scale models require clusters and pre-screened slices, not a dense J-Space fantasy.

Can I apply the protocol if I only have closed-source language model APIs?

Full J Lens requires gradient and activation access, which most closed APIs do not provide. Without internals you cannot compute Jacobians or run activation-level interventions, so the protocol cannot be applied in its mechanistic form. You can still write precise behavioral questions and minimal-pair datasets. Those results are behavioral interpretability. They are not J-Space analysis, and they should not be labeled as mechanistic interpretability of hidden model components.

How does J Lens relate to sparse autoencoders and feature dictionaries?

SAEs provide a basis that can make Jacobians readable by mapping hidden states into sparser coordinates with labels. Without a structured basis, a residual gradient is a wide vector with no semantics. With a dictionary, |J_i| can be discussed as the sensitivity of a named feature. The protocol can run in raw residual space; it is usually more interpretable in a dictionary. SAEs do not replace the lens curve, the contrast pair, or the intervention. A labeled feature with no causal score is still tourism.

Is the protocol specific to transformers, or can it handle other architectures?

Transformers and language models are the primary targets because of TransformerLens-style tooling and shared benches (IOI, induction, factual recall). The protocol is architecture-agnostic whenever gradients and internal activations are accessible. CNNs, mixers, state-space models, and RL world-model stacks can be instrumented. Non-transformer applications need custom hooks and often custom dictionaries. The core steps – J-Space definition, hypothesis formation, causal interventions, negative controls – do not change.

Where should a new researcher start?

Start with a small public transformer and a documented hook library. Reproduce an induction or IOI template: register the metric, compute per-head sensitivities, patch the winners, and run random-head controls. That exercise teaches the loop where ground truth is relatively crisp and compute is cheap. Then move to a mid-scale open model and add an SAE basis. Do not begin on a closed API or a 70B dense Jacobian. The scientific case for GPT-2-class first runs is that they falsify tooling bugs before they falsify theories of large models.

How should a misleading logit-lens curve change the causal design?

If an early layer already decodes the answer, treat that peak as a likely basis-change artifact unless a path-isolated intervention at that layer restores the clean-versus-corrupted gap and random sites do not. Shift confirmatory patches toward later writes implicated by the rest of the curve. If a late peak fails to patch, do not silently change layers after seeing the table; report the failure and test multi-hop paths or a bad metric. Lens tutorials that never touch this branch, and patching tutorials that ignore the lens curve, are how false circuits get published.

What does a local circuit on a synthetic task not prove about production safety?

It does not prove that the same mechanism runs in a production language model, that evals are adequate, that the model lacks other circuits for the same behavior, or that alignment-relevant properties (refusal, deception, hidden goals) have been audited. It does not show that the system has emergent causal world models in any sense relevant to consciousness debates. Under NISTIR 8312, knowledge limits are part of the explanation. A correct GPT-2-class diagram is a lab result. Promotion into an eval or a safety case needs held-out distributions, off-target measurements, and an explicit refusal to overfit the story to one template.

Editorial Team

Editorial Team

The J-Space Editorial Team curates and synthesizes cutting-edge research in mechanistic interpretability, AI alignment, and cognitive science. With a commitment to precision, empirical rigor, and clear exposition, the team distills complex technical developments into accessible yet authoritative analyses for researchers, engineers, and policymakers working at the frontier of artificial intelligence.