Sparse Autoencoders Use L1 Penalties to Isolate Distinct Model Features
Sparse autoencoders use L1 penalties on residual activations to generate sparse mostly zero latents that correspond to candidate features in language…
Category Deep Dives
Math behind J-lens/Jacobian, layer-specific behavior, experiments replicating on open models (Qwen, etc.), code/tutorials for probing J-space.
Latent Correctness Directions Emerge from Residual Activations Without Probes
Latent correctness directions appear in residual activations of language models when using Jacobian lenses and sparse autoencoders instead of probes.
Anthropic Interpretability Research Explained: Inside the Push for Transparent AI
In brief Anthropic's interpretability research employs dictionary learning with sparse autoencoders to extract monosemantic features from the internal…
What Does the Jacobian Lens Reveal About Claude’s Internal Representations?
The jacobian lens is a gradient-based method that identifies j space in Claude's internal representations, a sparse subspace accounting for less than 10%…