Abstract:We study the thermodynamic memory capacity of modern Hopfield networks (Dense Associative Memory models) with continuous states under geometric constraints, extending classical analyses of pairwise associative memory. We derive thermodynamic phase boundaries for Dense Associative Memory networks with exponential capacity $M = e^{\alpha N}$, comparing Gaussian (LSE) and Epanechnikov (LSR) kernels. For continuous neurons on an $N$-sphere, the geometric entropy depends solely on the spherical geometry, not the kernel. In the sharp-kernel regime, the maximum theoretical capacity $\alpha = 0.5$ is achieved at zero temperature; below this threshold, a critical line separates retrieval from non-retrieval. The two kernels differ qualitatively in their phase boundary structure: for LSE, a critical line exists at all loads $\alpha > 0$. For LSR, the finite support introduces a threshold $\alpha_{\text{th}}$ below which no spurious patterns contribute to the noise floor, and no critical line exists -- retrieval is perfect at any temperature. These results advance the theory of high-capacity associative memory and clarify fundamental limits of retrieval robustness in modern attention-like memory architectures.
| Comments: | Proceedings of the 43rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 |
| Subjects: | Disordered Systems and Neural Networks (cond-mat.dis-nn); Machine Learning (cs.LG) |
| Cite as: | arXiv:2604.07401 [cond-mat.dis-nn] |
| (or arXiv:2604.07401v2 [cond-mat.dis-nn] for this version) | |
| https://doi.org/10.48550/arXiv.2604.07401 arXiv-issued DOI via DataCite |
From: Tatiana Petrova [view email]
[v1]
Wed, 8 Apr 2026 09:21:22 UTC (61 KB)
[v2]
Wed, 6 May 2026 13:47:42 UTC (115 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。