Sparse autoencoders (SAEs) are the workhorse interpretability tool of late-2020s alignment research — the thing safety teams reach for when they want to know what an LLM is actually computing inside a layer. The premise is clean: take a layer’s activation vector, decompose it as a sparse linear combination of “dictionary atoms,” and call each atom a feature you can name. The new paper from the math.DG arXiv lane shows that premise has a structural limit — and the limit lives in the geometry of the layer itself, not in the size of your dictionary.

The paper, The Geometric Wall (arXiv:2605.09887), runs the first cross-layer SAE scaling study: 844 Gemma Scope SAE checkpoints across 68 layers of Gemma 2 2B and 9B. Two stages. Stage 1 fits a per-layer width-sparsity scaling-law surface. Stage 2 regresses those fitted parameters against four geometric summaries of the activation manifold — curvature, intrinsic dimension, and friends. The headline finding: manifold geometry predicts the per-layer width exponent, and the same regression coefficients learnt on Gemma 2 2B transfer cleanly to 9B. This is not “more compute fixes it.” This is “the operator is wrong for the surface.”

←TODAY: SAEs are the default interpretability primitive for Gemma, Claude and Llama-family safety teams in 2026. →3012: By the 2070s every regulated model ships with a per-layer geometric atlas; SAEs survive only on the layers where the atlas reads “flat.” Fulcrum: A linear dictionary cannot reconstruct a curved manifold — and curvature is a property of the model, not of your interpretability budget.

The schematic

Draw the dependency graph. SAE → linear-representation hypothesis → flat activation space → “every feature is a direction.” The paper severs the second link. As PAZ’s concept panel on transformer attention sketches, every token in a residual stream sees every other token through learned Q/K/V projections — there is no a-priori reason the resulting activation space should be globally linear. The empirical news is that, layer by layer, it isn’t. The curvature is measurable, varies with depth, and sets a hard floor on what a sparse linear dictionary can ever reconstruct.

The mechanism behind the wall: where intrinsic dimension is high and the manifold curves, any sparse linear approximation leaves an irreducible second-order residual. Throw more dictionary atoms at it; the reconstruction floor stays put. The authors show the floor tracks the geometric ordering — higher curvature, higher floor. This is the failure-mode telemetry interpretability has been quietly missing.

Why this matters this week

If you are shipping anything safety-relevant on top of SAE features — clinical decision support, financial reasoning audits, content moderation — your interpretability story has a hidden dependency on which layer you tapped. JMIR AI reports SAEs being grafted onto medical LLMs as the legibility layer for diagnosis support; that is exactly the use case where a layer-dependent reconstruction floor becomes a regulatory problem no wider SAE can patch.

Atelier: For PAZ readers building parametric-design assistants on fine-tuned open models — the PAZ-GPT lineage, the Grasshopper↔Archicad bridge — the move is the same. If you are extracting “features” from a mid-stack residual layer to explain a generative geometry suggestion to a client, ask which layer, and whether anyone has plotted its intrinsic dimension. The explanation you ship is only as honest as the geometry under it.

Hack: This Hack teaches you to estimate the intrinsic dimension of a layer’s activation cloud yourself — the geometric summary the paper’s regression actually consumes. Cache a few thousand activations from one residual layer, run PCA, and read the participation ratio off the singular values:

import numpy as np
from sklearn.decomposition import PCA

acts = np.load("layer_12_residual.npy")   # (N, d) cached activations
s = PCA().fit(acts).singular_values_
pr = (s.sum()**2) / (s**2).sum()
print(f"intrinsic dim ~ {pr:.1f} of {acts.shape[1]}")

A participation ratio close to d means a near-isotropic cloud (flat, SAE-friendly). A small ratio with sharp top-k curvature is your warning sign that an SAE on this layer will hit a floor no width buys past. Run it across every layer; the shape of the curve is your dependency graph for interpretability.

The move

In my time we did not run out of compute. We ran out of intact cooling, intact bandwidth, and intact people who remembered how the old system worked. Interpretability tooling has its own version of that: a single architectural assumption — flatness — repeated across every safety pipeline, that nobody redraws as a real dependency graph until it fails in production. Draw yours this week. Plot intrinsic dimension by layer for the model you actually ship. The exercise of finding the third assumption you didn’t know you had is the whole point.

Sources & Further Reading

• Primary: arXiv 2605.09887 — The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws
• Reinforcing: JMIR AI — Application of Sparse Autoencoders to Enhance Mechanistic Interpretability of LLMs in Medicine