Knowing Both Facts Still Fails the Composed Two-Hop Query

Fact-checked by the J-Space editorial team

Two-Hop Reasoning Circuits in Language Models

Updated September 11, 2026

Two hop reasoning is the smallest compositional case that still requires more than lookup: given two facts that were never taught as a pair, a model must bind a source entity to a bridge entity and then to an object. That is the core of two hop question answering, and it is the setting in which researchers can still hope to map internal mechanisms rather than only report benchmark scores. The one-hop substrate for this composition is now relatively well localized. Geva et al. give the mechanistic account of subject enrichment, relation extraction, and late-site object prediction that later two-hop papers build on, and prior work on feed-forward layers treats those MLPs as key-value memories.

This article reconstructs that circuit-level picture for AI researchers, interpretability engineers, and safety practitioners. It distinguishes one-hop recall, latent composition, written chain of thought, and A-to-C memorization; it summarizes residual-stream timing and bridge-entity binding; and it treats activation patching, causal tracing, and attribution graphs as evidence standards rather than as metaphors for thinking. It does not claim that “the circuit” for two hop reasoning is fully mapped in frontier large language models, and it does not treat next-token prediction as an explanation of composition.

Two hop reasoning circuits in language models: hop reasoning versus one-hop lookup

A hop, in this literature, is movement of a bound fact through the residual stream, not a HotpotQA score. One-hop recall retrieves an attribute of a subject that is already in the prompt or in a mid-layer memory (Alice lives in Paris). Two hop reasoning additionally requires an intermediate entity that is not the asked-for object: the model must instantiate Paris as a bridge and then read a second relation off that bridge (Paris is in France) in order to answer a composed query (In which country does Alice live?). Neither fact alone answers the question. Written chain of thought is a third computation: hop-1 and hop-2 are emitted as tokens, which changes both the available residual-stream sites and the training objective. Latent two hop reasoning is internal composition without those intermediate tokens. Conflating the three is the main reason prompting explainers and multi-hop leaderboards talk past circuit analysis. Two hop question answering on a benchmark can succeed by lookup, by a stored A-to-C clique, by lexical overlap, or by genuine composition; only the last is a two-hop circuit.

The bridge entity is the mechanistic bottleneck. In the residual stream it must be written at a token site and layer that later heads can read, then treated as a new subject. If the model never instantiates B and still outputs C, the path is a shortcut, not hop reasoning. If B becomes linearly decodable only after the layers that implement the second relation, hop-2 has nowhere to run. That is why investigating two hop reasoning is an information-flow problem before it is an accuracy problem. Large language models can be strong next-token predictors, and can even answer hop-1 and hop-2 in isolation, while lacking the binding and timing needed to compose them. Mixed evidence on two hop question answering is expected under that split: first-hop retrieval can scale with memorized facts in training data while second-hop composition remains fragile. Researchers studying llm reasoning therefore separate surface success from a source-to-bridge-to-target path.

Because natural text is saturated with co-occurrence, frequency cues, and memorized templates, artificial experimental setups are used to test two hop reasoning without a stored composite. Typical controls present two one-hop facts separately (source to bridge, bridge to target), use bridge tokens with no lexical overlap to source or target, add distractor chains so the model must select B among candidates, restrict the object to a single token so uniform guessing is measurable, and invent names or categories to block pretraining leakage. Those artificial experimental setups are not a claim that natural two hop question answering is uninteresting; they are a claim that natural items cannot isolate composition. Prior work on reading-comprehension benchmarks already showed that many items labeled multi hop reasoning were answerable from a single span. The same confound appears in two hop latent reasoning whenever facts co-occur in training data. Synthetic triplets with distractors, as used when investigating two hop reasoning in small transformers and in frontier evaluations of latent two hop reasoning, exist to remove that confound. Artificial experimental setups still leave open whether the recovered heads are the same ones used on natural text; that mapping is an empirical question for patching, not a reason to skip controls.

The experimental landscape from 2023 to 2025 is mixed evidence, not a phase transition in the popular sense that scale dissolved composition. Reasoning circuits constrain accuracy on complex reasoning tasks, and raw parameter count does not by itself implement binding. Small attention-only transformers trained on synthetic two-hop tasks with distractors show a sharp phase transition: attention is unstructured and accuracy sits near chance through a slow-learning period, then a sparse circuit appears and composition becomes reliable. Frontier evaluations of latent two hop reasoning, including controlled tests without chain of thought, often find that large language models fail to compose two independently learned facts even when each hop is answerable. Performance can improve when one fact is natural (present in pretraining) and the other is synthetic, which is a warning about leakage rather than a proof of llm latent reasoning capabilities. Across prompting, scale, and generic fine tuning, two hop question answering is only partly solved. Specialized fine tuning and chain of thought reasoning can raise token-level accuracy; whether that is llm latent reasoning or pattern completion over co-located facts is exactly what circuit analysis is for. Mixed evidence is therefore the honest summary: hop-1 recall is often present, hop-2 composition is often absent, and many complex agentic tasks inherit the same gap when they require unstated intermediate entities.

• Two one-hop facts are presented separately so the composite is not in context as a single clique.
• Bridge entities are distinct tokens, blocking lexical overlap shortcuts between source and target.
• Distractor facts force discrimination of the correct bridge rather than a frequency prior.
• Single-token answers make uniform guessing and random guessing distinguishable from composition.
• Synthetic names and invented categories reduce leakage from training data.

Internal mechanisms of factual recall in large language models

Before two facts can compose, each fact must live somewhere. The convergent account from independent labs is that mid-layer MLPs implement hop-1 key-value memories: a subject-enriched residual at the last subject token is used as a key, and the corresponding value writes an attribute into the stream. Feed-forward layers hold about two-thirds of transformer parameters, which is why they dominate factual association capacity (Geva, Schuster, Berant, Levy, 2021). Dissecting Recall of Factual Associations then splits one-hop inference into subject enrichment, relation extraction, and late-site object prediction. Causal tracing in the ROME project shows that those associations are localized enough for rank-one MLP edits to rewrite a fact. Linear relational embeddings and function vectors add a geometric operator view: many relations are approximately linear maps on residual states, which is the reusable hop-1 primitive a second hop would need. Underlying mechanisms at this stage are therefore not “the model knows France” in a diffuse sense; they are site-specific writes at the last subject token, relation-sensitive MLP directions, and a late readout. Language models that fail two hop reasoning often still succeed at these one-hop internals, which is why one-hop accuracy is not diagnostic of composition. Internal mechanisms of this form are the substrate, not the two-hop path.

That substrate is also why knowledge editing and two-hop circuits interact. If hop-1 is a localized MLP computation, an edit can change Alice’s city without changing the downstream heads that should treat the new city as a subject. Rank-one model editing remains the standard demonstration that one-hop factual associations are localized enough to rewrite; without that localization there would be no well-posed hop-1 write for a composition circuit to read. Linear decodability of relations does not by itself prove that language models compose two relations, but it gives a concrete mechanism they could use: apply R1 at mid layers, bind the decoded bridge, apply R2 later. Function vectors are one named instantiation of a reusable relation operator in activation space. None of this requires anthropomorphic language. It requires token sites, layer indices, and intervention outcomes. The underlying mechanisms are testable: patching the last-subject-token residual should restore hop-1; patching a later bridge site should be required for hop-2; editing the hop-1 MLP should change the bridge that hop-2 sees. When those tests fail, the model is not implementing the composition path even if two hop question answering is occasionally correct.

object ≈ Unembed( Read_late( Bind( MLP_mid(subject, r1) ), r2 ) )
# subject: last-subject-token residual; r1, r2: relation cues
# MLP_mid: hop-1 key-value recall; Bind: attention copy/pointer of the bridge
# Read_late: hop-2 extraction treating the bridge as a new subject

Reasoning circuits that implement hop reasoning through bridge entities

Once hop-1 recall exists, a two-hop path has three mechanically distinct stages. Hop-1 write: at the subject token, relation-sensitive MLPs and early heads enrich the residual with the first attribute and extract a candidate bridge. Bridge-entity binding: attention heads copy or pointer-bind that intermediate entity into a site later layers can read, often against distractors. Hop-2 read: later layers treat the bound bridge as a new subject and extract the second attribute toward the final token. In small attention-only models used to test two hop reasoning, this can appear as a three-layer sequence: layer-1 heads copy entity identities into buffer positions; layer-2 heads select the correct bridge among distractors via source-to-bridge query-key alignment; layer-3 heads route the target toward the output position. In full transformers the same roles are distributed across more heads and MLPs, but the information-flow skeleton is the same. Intermediate representations must align: if the bridge vector after binding does not match what hop-2 queries expect, composition fails even though both facts are stored. That alignment requirement is a reason generalization to novel bridges is fragile, and a reason two hop latent reasoning can look intact on memorized cliques while collapsing on new B tokens. Reasoning circuits of this form are sparse algorithmic structures, not a diffuse improvement in llm reasoning scores.

Residual-stream timing is part of the anatomy, not an optional visualization. The bridge must be written early enough for hop-2 read to consume it. Logit-lens style projections are used to ask when the bridge token is decodable versus when the object is decodable; in a working path the bridge appears in middle layers and the target only later. From a Jacobian-lens view, the same path is a structured map from e1 to e3 mediated by e2: if the local Jacobian of object logits with respect to the bridge residual at layer L is large and structured, layer L is participating in hop-2; if it is diffuse, the model is not using the bridge. Attribution graphs at production scale make the same claim as a feature-level graph: hop-1 write, bridge copy, hop-2 read as edges, not as a vibe that the model “thought step by step.” Induction-head and IOI circuits already show that transformers implement reusable copy/bind subcircuits; a two-hop path needs those primitives plus an MLP memory for each fact. Language models that implement the bind primitive still fail multi hop reasoning when the memory write is late, when distractors capture attention, or when hop-2 queries a different relation geometry than hop-1 produced. Circuit analysis that only names “reasoning heads” without token sites therefore under-specifies the path.

• Hop-1 write. Last-subject-token residuals and mid-layer MLPs retrieve the first attribute and propose the bridge.
• Bridge bind. Attention heads copy or pointer-bind B, suppressing distractor entities in the stream.
• Hop-2 read. Later layers treat B as subject, extract the second relation, and write the object toward the output token.
• Alignment constraint. The bound bridge vector must lie in the domain of the second relation operator; mismatch yields fluent but uncomposed answers.

Random guessing, reasoning shortcuts, and latent reasoning failures

Shortcut memorization of A to C is the central false positive for two hop reasoning. If the composite was in training data, or if A and C co-occur strongly, the model can output C without instantiating B. Dataset artifacts (lexical overlap, entity frequency, question templates) produce the same behavioral profile as composition. Reasoning shortcuts of that kind are why co-occurrence during pretraining is not evidence of llm latent reasoning capabilities. Patching is the discriminator: if ablating or replacing the bridge representation does not change the object logit, the model did not use B. Prior work on multi hop reasoning benchmarks showed that many items were single-span solvable; two hop question answering inherits that failure unless evaluations are shortcut-free. Reasoning shortcuts also include skipping the second relation entirely and jumping from a question pattern to a common answer, which circuit analysis of longer chains often finds. Mixed evidence on llm reasoning should be read with that taxonomy in hand: success given co-occurrence, failure given independently stored facts, and success given written hops are three different computations. Latent reasoning is only the case where B is internally bound and used. Reasoning shortcuts can still produce a confident final answer, which is why accuracy without interventions is not definitive evidence of composition.

A distinct failure mode is hopping too late. The bridge entity is resolved only after the layers that could have used it as a subject. Next-token prediction can then answer hop-1 and hop-2 as separate queries while failing the composed query: each fact is stored, but the forward pass does not make B available on time. Layer-wise information flow is a constraint, not just a description. This is one of the few mechanistic explanations of two-hop failures that does not reduce to “large language models only predict the next token,” which is too coarse to predict when composition works. In latent two hop reasoning evaluations, the bottleneck is often described as a role mismatch: B was trained as an output target (the answer to hop-1) but must serve as an internal input to hop-2. Two hop latent reasoning can therefore fail even when llm latent reasoning is probed with facts the model “knows.” Investigating two hop reasoning without measuring when B is decodable will mis-label this as a missing fact. Underlying mechanisms here are residual-stream availability and layer capacity, not motivation or “effort.” Distractor overload can push the same models back toward random guessing by lowering the signal-to-noise ratio of bridge selection, which is a routing failure rather than absence of hop-1 memories.

When we fine tune llms on synthetic two-hop data, behavior can reorganize abruptly rather than drift. A slow period of unstructured attention and near-chance accuracy can give way to sparse bridge-tracking heads; more distractors delay that phase transition by reducing signal relative to noise. That is a training-dynamics observation about when reasoning circuits become linearly separable in attention, not a guarantee that we fine tune llms into general multi hop reasoning. Fine tuning on synthetic facts can elicit two hop latent reasoning in a controlled setting while leaving reasoning shortcuts intact on natural text. OOD second hops, prompt rewording, and bridge-name collisions still break composition after the transition. First-hop retrieval often scales more cleanly with model size than second-hop composition across relation types. Safety-relevant failures follow the same anatomy: wrong entity linking, distractor documents capturing the bind step, and hallucinated bridges invented to connect two memorized facts. In those cases the model still produces a fluent object because hop-2 ran on the wrong B. Definitive evidence of a two-hop path has to survive bridge replacement, not merely beat chance after we fine tune llms on the evaluation distribution.

• Direct A-to-C memory. Composite stored; bridge ablation is silent.
• Frequency and template cues. Reasoning shortcuts that never select among distractor bridges.
• Hopping too late. B decodable only after hop-2 layers.
• Distractor capture. Bind heads route through the wrong intermediate entity.
• OOD second relation. Hop-1 geometry does not match the hop-2 operator.

Circuit analysis of multi hop reasoning and multi step reasoning

Behavioral accuracy cannot separate composition from cliques. The evidence standard is intervention. Activation patching (including causal tracing) replaces a residual at a site with the activation from a counterfactual prompt and measures the object logit; it localizes where information is sufficient. Path patching restricts the patch to a route through specified edges, asking whether hop-1 write, bridge copy, and hop-2 read are jointly necessary. Full attribution graphs, as in Anthropic’s Transformer Circuits work, expand that path story to feature-level nodes and causal edges in production-scale models. Tracing the Thoughts is the accessible write-up of planning, multi-step reasoning, and feature-level graphs inside a frontier model. These tools are how investigating two hop reasoning becomes a claim about graphs rather than about leaderboards. Probe correlations (linear decodability of B) are not equivalent: a feature can be readable and unused. Circuit analysis that stops at probes therefore cannot test two hop reasoning as composition. Language models may linearly encode many entities that never enter the hop-2 path. Definitive evidence requires that intervening on the proposed bridge site and heads changes the object in the direction predicted by the path, and that control sites do not.

What frontier attribution graphs add is scale and an explicit computational graph for facts that look multi-step from the outside. They are the primary public source for treating two-hop facts as assembled graphs rather than a single lookup, and they should be cited as testable structure (hop-1 write, bridge copy, hop-2 read), not as evidence that a system is conscious or that “Claude thinks.” Researchers studying llm reasoning can ask the same questions on open models with activation and path patching: which layer first writes B, which heads copy it, which later MLPs read it. Toy three-layer circuits remain useful because they make the phase transition in attention patterns inspectable; they do not automatically transfer head-by-head to hundred-layer large language models. The honest position is convergence on roles (memory write, bind, late read) plus incomplete maps at frontier scale. Circuit analysis of multi hop reasoning at that scale is still sparse relative to one-hop ROME-style maps. Prior work on induction heads and IOI is relevant because it already isolated copy/bind subcircuits; two-hop work reuses those primitives. Multi step reasoning in attribution graphs is not a different species from two hop reasoning; it is additional hops stacked on the same residual-stream constraints, including the risk of hopping too late at each extra bind.

Failure modes of the tools are part of the evidence standard. Superposition and polysemanticity mean a dimension can carry more than one feature, so a clean graph can be an incomplete projection. Incomplete ablations leave residual paths: destroying one bridge head may leave another, producing the false conclusion that the circuit was unnecessary. Patchscopes-style representation swaps can detect whether a pathway uses the bridge or a positional cue, but they inherit the same incompleteness if the swapped subspace is mixed. Jacobian magnitudes at a bridge site are a useful scalar for “is hop-2 coupled to B?” and still do not name every participating feature. None of these caveats license ignoring interventions in favor of accuracy. They license multiple consistent interventions, control paths, and an explicit refusal to say the circuit is fully mapped. For safety work, that incompleteness matters: an oversight procedure that trusts a partial graph can miss an alternate A-to-C path. Definitive evidence, in this setting, is always relative to a stated graph and a stated counterfactual family, not an exhaustive census of every head in large language models. Researchers studying llm reasoning should report negative controls (sites that should not matter) with the same status as positive patches.

Chain of thought, knowledge editing, and llm reasoning in language models

Latent reasoning is internal composition; chain of thought emits intermediate tokens. High accuracy with written hops does not imply llm latent reasoning capabilities. In strict synthetic settings, language models can sit at chance on two hop questions without CoT when supporting facts were learned independently, while the same items become easy once hop-1 is written into the context. That is expected if the bridge-entity bottleneck is real: emitting B as a token supplies a new subject site that hop-2 already knows how to use. CoT is then a workaround for a missing internal bind, not a window into it. Unfaithful chain of thought is the complementary case: the written hops do not match the internal path. A model can verbalize B1 while internally using a memorized A-to-C edge, or can internally bind B2 while writing a plausible but unused B1. Interpretability that only reads the tokens therefore cannot audit composition. Drawing conclusions from CoT traces without patching the claimed bridge is not an evaluation of two hop latent reasoning. The same split applies when drawing conclusions about agent behavior: many complex agentic tasks look like multi hop reasoning in the transcript while the forward pass uses shortcuts. Potentially enabling oversight of the actual path means tracing hop-1 write, bind, and hop-2 read, not scoring the transcript. Chain of thought remains useful as a crutch and as a communication channel; it is not the circuit.

Knowledge editing makes the split operational. A successful hop-1 edit (ROME-style rank-one write in an MLP) can change the stored city of Alice and still leave hop-2 broken if composition is a separate path, or if hop-2 still reads an unedited clique. Conversely, an edit that patches a shortcut A-to-C mapping may change composed answers without ever inserting a usable bridge. Editing coverage that stops at one-hop rewrite success therefore overstates reasoning capability on composed queries. Head ablation, function-vector edits, and targeted fine tuning of bind layers are the corresponding two-hop interventions: they test causal roles rather than only memory contents. If ablating a proposed layer-2 bind head collapses composition while leaving one-hop recall intact, that head is implicated in hop reasoning rather than in fact storage. Function-vector edits that redirect what a bridge maps to can change hop-2 targets without retraining. These are engineering handles on reasoning circuits, and they cut both ways: they can reduce entity-linking errors, and they can also be misused. Drawing conclusions about alignment from edit success requires checking the composed query, not only the edited one-hop prompt. Potentially enabling oversight in that setting means logging whether the post-edit bridge is the one hop-2 reads.

Scaling from two hops to n-hop and to tool-using agents reuses the same constraints. Each extra hop adds a bind that can happen too late, a distractor that can capture attention, and a chance to skip to a memorized endpoint. Many complex agentic tasks are n-hop in exactly this sense: an intermediate entity (a retrieved document, a tool result, a subgoal name) must be written into the stream in time for the next operator. If that intermediate is only verbalized, the system may depend on chain of thought the way two-hop models depend on writing B. If it is only latent, hopping-too-late predicts silent failures that look like competent llm reasoning until the tool argument is wrong. Reasoning capability on those tasks is therefore not a single scalar. It is whether each hop’s memory, bind, and read are in place, and whether reasoning shortcuts are suppressed when the composite was never trained. When we fine tune llms to emit tools or to answer composed questions, the fine tune can strengthen a sparse circuit or it can install a new clique. Distinguishing those outcomes is the same intervention program as for two hop reasoning. Complex tasks that require unverifiable intermediate entities are where missing circuits are most expensive, because the transcript can still look like multi hop reasoning.

Open problems follow from the evidence rather than from a missing slogan. Circuits recovered in toy transformers need role-level validation in frontier large language models; distributed implementations may preserve write/bind/read without preserving a three-head cartoon. Phase-transition timing still lacks a settled theory in terms of signal-to-noise, head capacity, and distractor count. Bridge-entity bottlenecks need a formal account in residual geometry and tokenization, not only a qualitative role mismatch. Attribution graphs must be cheaper and more complete before they can be used as routine oversight. Global-workspace language should stay metaphorical: attention that broadcasts a bridge across positions is measurable signal flow, not a verdict on consciousness, and this article takes no position on whether language models are conscious. J-Space style Jacobians are one way to quantify coupling from bridge residuals to object logits without claiming a new empirical result here. The position that is defensible is narrower: independent labs converge on mid-layer MLPs as the hop-1 store; causal interventions support a path story; linear relational structure supplies a composition operator; hopping too late predicts a specific failure; copy/bind subcircuits already exist. That is enough to reject both “LLMs cannot compose anything” and “the two-hop circuit is fully mapped.” It is also enough, potentially enabling oversight, to demand bridge-level tests before treating two hop question answering as llm latent reasoning. Complex reasoning tasks will keep looking solved in aggregate while remaining unsolved as circuits until those tests are standard. Reasoning capability, on this view, is a property of a path in a forward pass, not of a leaderboard or of a fluent chain of thought.

• Scaling maps. Confirm write/bind/read roles beyond toy depth without claiming a complete frontier census.
• Edit composition. Require hop-2 evaluations after hop-1 rewrites.
• Faithfulness tests. Compare verbalized hops to patched internal bridges.
• n-hop timing. Measure whether each extra bridge is available before the next operator, including in many complex agentic tasks.

FAQ

How is two hop reasoning different from next-token prediction and from written chain of thought?

Next-token prediction is the training objective. Two hop reasoning is an internal algorithm that binds a bridge entity and reads a second relation. Written chain of thought supplies hop-1 as tokens, which can bypass a missing latent bind. A model can be a strong next-token predictor, including on two hop question answering with CoT, while lacking latent two hop reasoning. The diagnostic is whether B is used in the residual stream in time for hop-2, not whether the object string is correct.

Does scaling large language models automatically produce llm latent reasoning?

Scale increases stored facts and often improves isolated hop-1 retrieval. It does not automatically install bind-and-read timing for hop-2. Latent two hop reasoning without CoT or fact co-occurrence remains fragile in controlled evaluations, which is why llm reasoning on composed queries can lag one-hop recall. Reasoning capability here tracks path completeness, not parameter count alone.

How should researchers test two hop reasoning without counting shortcuts?

Use shortcut-free items, independently stored facts, and interventions on the proposed bridge. Test two hop reasoning with distractors, no lexical overlap, and path patching through B. If bridge replacement does not move the object logit, the success was not composition. That protocol is how to test two hop reasoning as a circuit rather than as a dataset score, and it is how researchers studying llm reasoning can compare latent two hop reasoning to written hops.

What does hopping too late mean, and why does it matter for two hop latent reasoning?

Hopping too late means the bridge becomes available only after layers that implement the second relation. Both facts can be known; the composed query still fails. Two hop latent reasoning is exactly the setting where that timing shows up, because no intermediate token recreates a subject site. It is a forward-pass constraint on llm latent reasoning, not a missing fact in the corpus.

If a ROME edit fixes hop-1, why can hop-2 still fail?

ROME rewrites a localized one-hop MLP association. Composition requires a separate bind-and-read path that may still point at an unedited clique or at a late, unusable bridge. Knowledge edits that ignore hop-2 can look successful on the edited fact while leaving two hop reasoning broken. Checking the composed query is part of the edit evaluation, not an optional extra.

What counts as definitive evidence of reasoning circuits rather than a probe correlation?

Linear decodability of a bridge is not sufficient. Definitive evidence is a predicted change in the object logit after activation or path patching of the proposed write, bind, and read sites, with control sites silent, plus a check that A-to-C shortcuts are not doing the work. Attribution graphs add feature-level edges for the same path. Incomplete ablations and superposition still limit how far any one graph can be trusted, which is why potentially enabling oversight depends on repeating interventions rather than on a single visualization of llm latent reasoning capabilities.

Editorial Team

Editorial Team

The J-Space Editorial Team curates and synthesizes cutting-edge research in mechanistic interpretability, AI alignment, and cognitive science. With a commitment to precision, empirical rigor, and clear exposition, the team distills complex technical developments into accessible yet authoritative analyses for researchers, engineers, and policymakers working at the frontier of artificial intelligence.