Mechanistic Interpretability of Atmospheric Rivers in GraphCast
AI weather models now rival operational forecasts, yet how they represent the atmosphere internally remains an open question. Feature attribution shows which inputs matter, not what the model computes. This work trains standard and Matryoshka sparse autoencoders (SAEs) on GraphCast, using atmospheric rivers as the phenomenon of focus, and finds that GraphCast computes atmospheric river intensity (integrated vapor transport) as a stable internal variable, even though it is neither an input nor a target. The Matryoshka SAE also orders concepts by importance and exposes their relations. These concepts persist across depth, and direct interventions confirm they are causal. The method identifies which internal variables a model actually relies on, a prerequisite for asking whether they remain meaningful as the climate warms.
Data Efficiency in Sparse Autoencoders
Sparse autoencoders (SAEs) are typically trained on hundreds of millions to billions of activation tokens, but little work has tested whether that scale is necessary. This work trains four SAE architectures on three language models, with budgets from 5M tokens up to a 500M-token reference, and measures two kinds of convergence: functional behavior on standard evaluations, and one-to-one reproducibility of learned features. How much data an SAE needs depends strongly on the metric. Reconstruction often approaches the full-data reference at much smaller budgets, while feature reproducibility keeps improving with more data. For Gemma-2-2B BatchTopK, strictly matched features rise from 2% at 5M tokens to 29% at 50M. Data selection matters too: random sampling better preserves the activation distribution and generally favors reconstruction, while diversity-based sampling often improves AutoInterp. A theoretical analysis separates statistical consistency of subsampled training from the stronger conditions needed for one-to-one feature recovery. Together, the results show SAE training can be far more data-efficient when budgets match the goal and feature-level claims are backed by explicit reproducibility checks.
