Abstract:To reduce LLM costs and latency, semantic caching systems must accurately identify when a new prompt matches a cached one. Current methods often rely on simplistic similarity measures, which limit their effectiveness. We introduce MVR-cache, a novel semantic caching approach that significantly improves retrieval accuracy by integrating Multi-Vector Retrieval (MVR). MVR-cache is built upon a learnable segmentation model that intelligently splits prompts, enabling fine-grained similarity comparisons via MaxSim. We derive the model's training objective from a rigorous theoretical analysis. This can ensure that optimizing this objective directly maximizes cache hits under strict correctness constraints. To solve the resulting non-differentiable combinatorial optimization problem, we leverage a reinforcement learning-based training strategy with the theoretically grounded objectives as the reward. Experimental results on established benchmarks across diverse tasks confirm that in comparison to the state-of-the-art, MVR-cache consistently increases the cache hit rates by up to 37% while maintaining the same correctness guarantees. MVR-cache is available at this https URL
| Comments: | Published in ICML 2026 |
| Subjects: | Information Retrieval (cs.IR); Databases (cs.DB); Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.24914 [cs.IR] |
| (or arXiv:2605.24914v1 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2605.24914 arXiv-issued DOI via DataCite (pending registration) |
From: Ali Noshad [view email]
[v1]
Sun, 24 May 2026 07:33:46 UTC (7,386 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。