Abstract:As large language models (LLMs) grow in scale and specialization, routing--selecting the best model for a given input--has become essential for efficient and effective deployment. While recent methods rely on complex learned routing strategies, their dependence on disparate training data and evaluation setups makes comparison and generalization difficult. In this work, we revisit LLM routing through the lens of simplicity. We show that a well-tuned k-Nearest Neighbors (kNN) approach not only matches but often outperforms state-of-the-art learned routers across diverse tasks. To support systematic evaluation, we introduce a suite of standardized routing benchmarks spanning instruction-following, question-answering, and reasoning tasks, as well as the first multi-modal routing dataset involving visual inputs. Our findings reveal that the locality properties of model performance in embedding space enable simple non-parametric methods to achieve strong routing decisions with lower sample complexity than parametric approaches. This challenges the prevailing trend toward sophisticated architectures and highlights the importance of thoroughly evaluating simple baselines before investing in complex solutions. To support reproducibility and further exploration, we will release all benchmarks and code upon publication.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2505.12601 [cs.LG] |
| (or arXiv:2505.12601v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2505.12601 arXiv-issued DOI via DataCite |
From: Yang Li [view email]
[v1]
Mon, 19 May 2025 01:33:41 UTC (354 KB)
[v2]
Thu, 14 May 2026 18:08:42 UTC (628 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。