



























Abstract:In this work, we developed and tested 3 techniques for vector quantization (VQ) based model weight compression. To mitigate codebook collapse and enable end-to-end training, we adopted cosine similarity-based assignment. Building on ideas from attention-based formulations in Differentiable K-Means (DKM), we further improved this approach by using cosine similarity for assignment combined with top-1 sampling and a straight-through estimator, thereby eliminating the need for weighted-average reconstruction. Finally, we investigated the use of differentiable neural architecture search (NAS) to adaptively select layer-wise quantization configurations, further optimizing the compression process. Although our method does not consistently outperform existing approaches across all quantization levels, it provides useful insights into the design trade-offs and behaviors of VQ-based model compression methods.
| Subjects: | Machine Learning (cs.LG); Hardware Architecture (cs.AR) |
| Cite as: | arXiv:2604.23172 [cs.LG] |
| (or arXiv:2604.23172v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.23172 arXiv-issued DOI via DataCite (pending registration) |
From: Terry Gou [view email]
[v1]
Sat, 25 Apr 2026 06:55:04 UTC (866 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。