






















Abstract:To lower the expertise barrier in machine learning, the AutoML community has focused on the CASH problem, which jointly automates algorithm selection and hyperparameter tuning. While traditional methods like Bayesian Optimization (BO) struggle with cold-start issues, Large Language Models (LLMs) can mitigate these through semantic priors. However, existing LLM-based optimizers generalize poorly to high-dimensional, structured CASH spaces. In this paper, we propose LB-MCTS, a trajectory-structured optimization framework that uses a Monte Carlo Tree Search tree as a shared state for algorithm selection, hyperparameter refinement, and BO-LLM proposer synergy. Within this shared state, BO provides algorithm-specific surrogate modeling for quantitative search, while the LLM exploits path-aware selective memory to generate semantic proposals and reflections. As the surrogate model improves, a reliability-aware proposer policy adaptively shifts from LLM-driven to BO-driven proposals within a unified search trajectory. Experiments on 104 AMLB datasets demonstrate that LB-MCTS consistently outperforms BO-based, LLM-based, and hybrid baselines.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2601.12355 [cs.LG] |
| (or arXiv:2601.12355v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2601.12355 arXiv-issued DOI via DataCite |
From: Beicheng Xu [view email]
[v1]
Sun, 18 Jan 2026 11:26:31 UTC (3,841 KB)
[v2]
Wed, 6 May 2026 21:35:23 UTC (3,923 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。