BARQ: Balanced Codebook Refinement for Low-Bit LLM Quantization
Improve low-bit quantization by giving codewords balanced opportunities to learn during fitting.
The short version
BARQ studies codeword concentration during vector quantization. When many weight blocks collapse onto a small subset of codewords, nominal capacity can be poorly used. The method introduces balanced soft assignments through optimal transport, combines them with a curvature-aware cost, and refines codewords using barycentric updates.
The balance constraint guides codebook fitting; deployment uses ordinary hard assignments and codeword lookup. This separates the geometry of finding a useful compressed representation from the format used to store and execute it.
Low-bit compression depends on how a model uses its discrete codebook, as well as how many bits it stores.
Balancing the fitting process
Curvature-aware transport, balanced assignment, and barycentric updates refine the codebook before hard deployment.
View in the paperWhy this matters
Nearest-codeword fitting can give some codewords too little updating mass. An underused codebook wastes representational capacity precisely when low-bit deployment makes that capacity scarce.
How it works
- Solve entropically regularized optimal transport with uniform marginals and curvature-weighted reconstruction costs.
- Update codewords using assignment-weighted barycentres, with fixed-assignment optimality conditions.
- Discard fitting-time soft assignments and encode weights with standard hard nearest-codeword assignments.
Does the codebook become less concentrated?
The maximum codeword share tracks how strongly weight blocks concentrate on the most-used codeword.
View in the paperWhat the study finds
The paper establishes sufficient conditions for improved hard-quantization distortion and evaluation loss. Across the evaluated LLMs and comparable bit budgets, it reports lower perplexity and higher mean zero-shot accuracy than the compared baselines.
Paired quality comparisons
The original table compares perplexity and average accuracy with GPT-VQ over multiple model and quantization configurations.
View in the paperWhat this means
Compression can therefore be understood as an allocation problem: a limited set of discrete representatives must cover weight structure according to its effect on the model. Assignment geometry becomes an additional design variable alongside precision and block size.
The practical question is whether better use of a fixed codebook translates into retained capability. BARQ evaluates that link with codeword-use diagnostics and paired model-quality comparisons, making the mechanism and its downstream value visible in the same study.
Paper and resources
This is an introduction to the linked paper. Refer to the original for method definitions, evaluation settings, and complete results.
Cite this work
@misc{barq,
title = {BARQ: Balanced Codebook Refinement for Low-Bit LLM Quantization},
author = {Cui, Chenhang and Xie, Xu and Xu, Linrui and Liu, Xiaohao and Zhu, Xingyu and Shen, Fei and Chua, Tat-Seng},
year = {2026},
eprint = {2610.04490},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.04490}
}