Abstract:Neural processes are meta-learning models that map context sets to predictive distributions. While inspired by stochastic processes, NPs do not generally satisfy the Kolmogorov consistency conditions required to define a valid stochastic process. This inconsistency is widely acknowledged but poorly understood. Practitioners note that NPs work well despite the violation, without quantifying what this means. We address this gap by defining the conditioning consistency gap, a KL divergence measuring how much a conditional neural process's (CNP) predictions change when a point is added to the context versus conditioned upon. Our main results show that for CNPs with bounded encoders and Lipschitz decoders, the consistency gap is $O(1/n^2)$ in context size $n$, and that this rate is tight. These bounds establish the precise sense in which CNPs approximate valid stochastic processes. The inconsistency is negligible for moderate context sizes but can be significant in the few-shot regime.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2604.19312 [cs.LG] |
| (or arXiv:2604.19312v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.19312 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | TMLR 2026 |
From: Robin Young [view email]
[v1]
Tue, 21 Apr 2026 10:20:19 UTC (1,445 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。