Abstract:Humans can pursue a near-infinite variety of tasks, but typically can only pursue a small number at the same time. We hypothesize that humans leverage experience on one task to preemptively learn solutions to other tasks that were accessible but not pursued. We formalize this idea as Multitask Preplay, a novel algorithm that replays experience on one task as the starting point for "preplay" -- counterfactual simulation of an accessible but unpursued task. Preplay is used to learn a predictive representation that can support fast, adaptive task performance later on. We first show that, compared to traditional planning and predictive representation methods, multitask preplay better predicts how humans generalize to tasks that were accessible but not pursued in a small grid-world, even when people didn't know they would need to generalize to these tasks. We then show these predictions generalize to Craftax, a partially observable 2D Minecraft environment. Finally, we show that Multitask Preplay enables artificial agents to learn behaviors that transfer to novel Craftax worlds sharing task co-occurrence structure. These findings demonstrate that Multitask Preplay is a scalable theory of how humans counterfactually learn and generalize across multiple tasks; endowing artificial agents with the same capacity can significantly improve their performance in challenging multitask environments.
| Subjects: | Machine Learning (cs.LG); Neurons and Cognition (q-bio.NC) |
| Cite as: | arXiv:2507.05561 [cs.LG] |
| (or arXiv:2507.05561v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2507.05561 arXiv-issued DOI via DataCite |
From: Wilka Carvalho [view email]
[v1]
Tue, 8 Jul 2025 00:55:47 UTC (5,035 KB)
[v2]
Mon, 4 May 2026 20:02:44 UTC (44,921 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。