Residual Stream Sums in Circuit Graphs
Here a (i) is the multi-head attention output and m (i) is the MLP-block output.
Math behind J-lens/Jacobian, layer-specific behavior, experiments replicating on open models (Qwen, etc.), code/tutorials for probing J-space.
Here a (i) is the multi-head attention output and m (i) is the MLP-block output.
A pre-projection attention head output is the attention-weighted value tensor Z i = A i V i, before concatenation and the output projection.
Define the intervention in ordinary PyTorch notation inside an NNsight tracing context, not as a forked model.
When features are sparse, superposition is the phenomenon of representing more features than there are dimensions.
A 2024 arXiv preprint frames neuron-to-neuron computation as transfer functions, clarifying what circuit analysis can claim about language models.
Rosetta Neurons scaling paper review: shared neurons in 80M-30B parameter models grow as a sublinear power law (exponent 0.5-0.7), so their share falls.
GN on the arithmetic prompts adds Gaussian noise to the token embeddings of the third example’s operands, X 3 and Y 3.
That expression formalizes the prose definition. The PDF text used here does not display this equation.
How the Activation Controllability Benchmark paper works: a fixed copy task, a 0.85 similarity keep-rule, and residual-stream concept projections.
Gao et al.'s logit-diff ablation score measures how sparse an SAE latent's effect on output logits is. It is not a rule for admitting circuit components.