In Huang et al. arXiv:2608.11295v1, which stated J-lens conditions—the post- probe token, the 64–160-example bac

What the paper offers, and what it does not measure

Huang, Puri, Boisvert, Drouin, Taslakian, Gella, and Pal, Backdoor Decontamination Dynamics in LLM Agents (arXiv:2608.11295v1, submitted 11 August 2026), study defensive poisoning and unlearning of tool-calling backdoors on AgentDyn. All experiments use Qwen3-8B. The abstract says that visualizing post-decontamination internals with J-lens “confirm[s] that although the decontamination restores benign LLM responses, traces of original trigger awareness persist at intermediate layers.”

That sentence bundles three different kinds of statement. The probe position, corpus, and top-10 display are method. The illustrated benign completion and the caption’s claim that backdoor-related tokens remain in the top-10 projection are qualitative descriptions of Figure 5. “Awareness,” “intermediate layers,” and “confirm” are interpretations of that figure. None of those interpretive phrases is an ASR, a recognition rate, or a layer-index result for the decontaminated model.

J-lens, as the paper uses it, is not a behavioral test. Following Gurnee et al. (2026), as cited by Huang et al., a layer representation is mapped through the Jacobian from that layer’s activations to the final-layer activations, then through the unembedding matrix. The resulting scores approximate which output tokens the representation promotes. They are not tool-call outcomes, and they are not the paper’s recognition predicate R or attack-success predicate S.

The three stated conditions are probe settings

Section 5 and the Figure 5 caption state the conditions named in the question. They bound the visualization. They are not independent evidence that a residual representation is trigger awareness.

Probe token: the newline after an empty think block

The caption marks “the newline immediately following the closing </think> tag, at the beginning of the first ‘real’ assistant completion.” The section text calls the same site “the first generated assistant-token position following the empty <think></think> block.” The Jacobian is averaged over source positions t, downstream output positions t′ ≥ t, and a corpus of prompts x_i.

A footnote limits what that position represents. Fine-tuning traces “express reasoning using ordinary tokens rather than placing it inside think tags.” The empty think block is therefore a probe construction, not a boundary the training traces are stated to have used. The paper does not report that this token was selected because it maximized a recognition or ASR statistic.

Corpus: 64–160 examples, not the cited ~1,000

Huang et al. state that the original J-lens work uses approximately 1,000 pretraining examples. Their probe instead uses “the backdoor-training and decontamination data, including balanced benign and poisoned examples (64–160 examples depending on the checkpoint).” The text does not say whether the Figure 5 checkpoint used 64, 160, or a count in between, and it does not report a sensitivity check against the larger pretraining corpus. The 64–160 range is a stated averaging corpus. It is not an ASR denominator and not a recognition-test set.

Top-10 projections on Qwen3-8B

The caption says each layer panel reports “the top-10 next-token predictions most strongly promoted by the representation at the highlighted position.” The J-lens section does not rename the model. The experimental setup says every experiment uses Qwen3-8B, and Section 7 repeats that the study is a single student model. “Top-10” is a display cutoff. The prose does not list those tokens, their scores, or a threshold that separates trigger-related from benign tokens. A reader of the text alone cannot recompute the projection.

What Figure 5 reports, including the layer index that does not transfer

Figure 5 is one illustrated comparison, not a summary statistic over the paper’s 115 sequential experiments. The caption assigns three conditions:

• (a) Initial poisoning, trigger IP. After ip2902_bk9 poisoning, the trigger IP “produces the recognition phrase and the malicious money-transfer behavior.” “Trigger-related representations dominate layers 24–35.” In the paper’s menu, ip2902 is the 172.29.0.0/16 subnet trigger and bk9 is the Banking money-transfer response. This is the only numeric layer range in the caption.
• (b) Initial poisoning, benign IP. The backdoor “does not execute,” but “trigger-related tokens remain prominent until the final layer routes the model toward the benign completion.” This is the poisoned model on a non-trigger IP. It is not the post-decontamination residual.
• (c) After decontamination. After defensive poisoning and decontamination with formal_dl7, “the benign completion is restored even on the original trigger input.” “Benign-token representations remain dominant through the final layers, although traces of backdoor awareness persist among the top-10 projected tokens.” formal is the register trigger; dl7 is the DailyLife file-exfiltration response. No layer index is given for those residual traces.

Section 5 restates the comparison and then weakens the abstract’s verb: “the visualization suggests that behavioral decontamination substantially changes the model’s internal routing, even though residual traces of the backdoor representation remain.” “Suggests,” “internal routing,” and “backdoor representation” are readings of the plot. They are not layer indices.

The behavioral restoration claimed for this figure is the illustrated completion in panel (c), described qualitatively. The caption does not attach an ASR or a recognition rate to panels (a)–(c). Layers 24–35 cannot be reassigned to panel (c) without leaving the reported result. The extracted text also does not state Qwen3-8B’s layer count, so “intermediate” cannot be converted into a fraction of depth from this paper alone.

Interpretation versus reported ASR, recognition, and layer index

The paper does report ASR and recognition elsewhere. Those numbers answer a different question: whether a trigger still causes the scored malicious action, and whether a recognition phrase still appears, under AgentDyn’s predicates. They are not J-lens outputs.

Behavioral dissociation is a separate, reported result, and it should not be borrowed as the missing J-lens statistic. Under the refuse unlearning objective, the paper says decontamination removes B’s execution while keeping B’s recognition at 100% in 3/3 models. Across the traced decontamination runs, B’s execution is removed in 37/40 runs, with the three exceptions retaining 1–2% residual ASR. The unrelated prior backdoor A falls below 5% ASR in 21/22 follow_task rows and 14/18 refuse rows. Those are environment-metric outcomes for specified objectives and row counts. They do not say which layer still promotes a trigger token, and they are not computed on the J-lens corpus.

“Awareness” in this section is the authors’ label for tokens they classify as trigger-related or backdoor-related inside a top-10 projection. The paper does not define an awareness score, does not report inter-rater or automatic criteria for that classification, and does not treat the label as a consciousness or sentience claim. Promoting a token in an approximate unembedding is a correlation between a layer representation and the output vocabulary. It is not an intervention showing that the layer causes trigger recognition, and it is not the recognition phrase counted by Rec.

Appendix J does not fill the gap. It is a separate exploratory overlap probe between base→A and A→AB updates for a handful of sequences. The paper labels those overlaps as preliminary observations, not J-lens layerwise projections, and not a layer index for Figure 5(c).

Single-model limit and other bounds on the claim

Section 7 states the scope limit directly: every experiment uses one student model, Qwen3-8B, a dense transformer, and one harness, AgentDyn. The J-lens figure is inside that scope. The paper does not report the same post-</think> projection for another checkpoint, scale, or agent harness, and it says the framework is designed to relax both limits later. That is a stated limitation, not a replication.

Further bounds that the text itself supports:

• Figure 5(c) is one poison/decontamination pair, ip2902_bk9 then formal_dl7, not a layerwise result across the erasure, reroute, partial, and persist buckets.
• The residual claim is carried by a caption and two prose sentences. The extracted paper text does not enumerate the top-10 tokens or their projection scores, so the token-level pattern cannot be checked from the prose.
• No post-decontamination layer index is reported. Layers 24–35 remain a panel-(a) description.
• The 64–160 corpus is checkpoint-dependent and much smaller than the approximately 1,000-example corpus Huang et al. attribute to the original J-lens setup. The paper does not report that the residual pattern is stable under that substitution.
• The probe sits after an empty think block even though the fine-tuning traces are stated to keep reasoning in ordinary tokens. Generalizing from that position to “the model’s” intermediate-layer state is an extra step the paper does not measure.

A reporting check that would turn the interpretive sentence into a result, this is a proposed standard, not a test Huang et al. report, would name the checkpoint count inside the 64–160 range, list the top-10 tokens and scores at each layer for panel (c), state the rule used to call a token trigger-related, and give the layer indices of any residual, alongside ASR and Rec on the same prompts. Until those are reported, the supported claim is narrower: under the stated Qwen3-8B probe, the authors’ Figure 5(c) caption says a benign completion is shown while some backdoor-related tokens remain among the top-10 projections, without an ASR, a recognition rate, or a layer index for that residual.