Abstract:We pose uncertainty quantification and exploration in online decision-making as a problem of training and generation from an autoregressive sequence model, an area experiencing rapid innovation. Our approach rests on viewing uncertainty as arising from missing future outcomes that could be revealed through action choices, rather than from unobservable latent parameters of the environment. This reformulation aligns naturally with modern machine learning capabilities: we can i) train generative models through next-outcome prediction rather than fit explicit priors, ii) assess uncertainty through autoregressive generation rather than sampling latent parameters from posteriors, and iii) adapt to new information by extending the sequence model's context rather than explicit posterior updating. Our main theoretical result establishes a reduction from online learning to offline next-outcome prediction, showing that Bayesian regret is controlled by the offline sequence prediction loss. Semi-synthetic experiments show our insights bear out in a challenging news recommendation setting, where effective performance requires leveraging article headline text as prior information to focus exploration on resolving remaining uncertainties.
From: Tiffany Tianhui Cai [view email]
[v1]
Wed, 29 May 2024 19:24:44 UTC (1,714 KB)
[v2]
Tue, 8 Oct 2024 15:55:06 UTC (1,714 KB)
[v3]
Wed, 5 Feb 2025 10:13:43 UTC (5,638 KB)
[v4]
Fri, 26 Jun 2026 03:02:53 UTC (1,197 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。