Abstract:Powerful large language models (LLMs) from different providers have been expensively trained and finetuned to specialize across varying domains. In this work, we introduce a new kind of Conductor model trained with reinforcement learning to automatically discover powerful coordination strategies among LLMs. Our Conductor learns not only to design targeted communication topologies for effective agent-to-agent collaboration, but also to prompt engineer focused instructions to the LLMs to maximally leverage their individual capabilities. We show that, by learning optimal coordination strategies over pools of powerful worker LLMs, a 7B Conductor achieves significant performance gains beyond any individual worker, attaining state-of-the-art results in challenging reasoning benchmarks, such as LiveCodeBench and GPQA. By training with randomized agent pools, our conductor effectively adapts to arbitrary sets of open- and closed-source agents, meeting any user requirements. Furthermore, allowing the Conductor to select itself as a worker gives rise to recursive topologies, elevating performance with a new form of dynamic test-time scaling through online iterative adaptation. More broadly, ours is among the early work demonstrating language model coordination can be unlocked through RL, where powerful coordination strategies emerge naturally in LLMs through pure end-to-end reward maximization.
| Comments: | To appear at the 14th International Conference on Learning Representations (ICLR 2026) |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2512.04388 [cs.LG] |
| (or arXiv:2512.04388v5 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2512.04388 arXiv-issued DOI via DataCite |
From: Stefan Nielsen [view email]
[v1]
Thu, 4 Dec 2025 02:23:13 UTC (3,426 KB)
[v2]
Thu, 29 Jan 2026 02:24:13 UTC (3,426 KB)
[v3]
Fri, 20 Feb 2026 14:32:50 UTC (3,426 KB)
[v4]
Sun, 1 Mar 2026 23:52:58 UTC (3,426 KB)
[v5]
Wed, 6 May 2026 14:06:21 UTC (3,425 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。