DV-Lens: Revealing the Functional Organization of Language Model Parameters
A vocabulary-space view of native parameters, tested through local output changes and targeted interventions.
The short version
DV-Lens treats the vocabulary as a shared coordinate system for interpreting native parameters. By following the downstream computation, it translates parameter directions into signed local output responses: which tokens a direction tends to promote, and which it tends to suppress. This connects the internal organization of attention and feed-forward modules to a common, testable interface.
The study then asks whether that functional organization contains information about model capability. DV-Complexity summarizes variation within functionally grouped parameters. Finite-difference checks, targeted interventions, and a comparison across 48 models connect the proposed readouts to observable behavior at different scales.
An explanation of a parameter becomes useful when it predicts a change in the model’s output.
From parameters to vocabulary responses
The downstream mapping distinguishes DV-Lens from direct vocabulary projection and activation decomposition.
View in the paperWhy this matters
A parameter interpretation becomes more useful when it predicts what will change in the output. The challenge is to connect parameters from different modules to a common, verifiable output space.
How it works
- Estimate downstream Jacobians for attention Q/K/V/O projections and FFNs over reference prompts.
- Project native parameter columns into vocabulary space to obtain signed local readouts.
- Group columns by their readouts and measure within-group structural variation with DV-Complexity.
Testing the local prediction
Finite-difference measurements test direction agreement and the recovery of high-impact vocabulary coordinates across nine models.
View in the paperWhat the study finds
Finite-difference tests report 98.0% coordinate-orientation agreement across 720 cases from nine models. Readouts also guide ablation, steering, and swapping across 21 models. Across 48 models, joint-parameter DV-Complexity correlates with capability rankings at Spearman ρ = 0.904.
Organization and capability
Across the fixed 48-model cohort, the plots compare DV-Complexity ranks with benchmark capability ranks. The relationship is correlational.
View in the paperWhat this means
The broader opportunity is to make interpretability predictive. A parameter map can support specific hypotheses about editing, steering, or transferring behavior when its readouts survive intervention tests. That makes the explanation part of an experimental workflow, rather than an endpoint.
DV-Complexity also suggests that capability may depend on how parameters are organized, alongside how many parameters a model contains. Its correlation with benchmark rankings motivates controlled studies of that relationship; the reported association alone does not establish that increasing this measure will improve a model.
Paper and resources
This is an introduction to the linked paper. Refer to the original for method definitions, evaluation settings, and complete results.
Cite this work
@misc{dv_lens,
title = {DV-Lens: Revealing the Functional Organization of Language Model Parameters},
author = {Cui, Chenhang and Yu, Jian and Miao, Shuyi and Liu, Xiaohao and Huang, Rui and Shen, Fei and Zhang, An and Chua, Tat-Seng},
year = {2026},
eprint = {2610.04489},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.04489}
}