


























Authors:Hua-Dong Xiong (1), Li Ji-An (2), Jiaqi Huang (3 and 4), Robert C. Wilson (1 and 5), Kwonjoon Lee (4), Xue-Xin Wei (6) ((1) School of Psychological and Brain Sciences, Georgia Tech, (2) Department of Psychology, New York University, (3) Department of Cognitive Science, Indiana University Bloomington, (4) Honda Research Institute, (5) Center of Excellence for Computational Cognition, Georgia Tech, (6) Departments of Neuroscience and Psychology, The University of Texas at Austin)
Abstract:Intelligent systems must maintain and manipulate task-relevant information online to adapt to dynamic environments and changing goals. This capacity, known as working memory, is fundamental to human reasoning and intelligence. Despite having on the order of 100 billion neurons, both biological and artificial systems exhibit limitations in working memory. This raises a key question: why do large language models (LLMs) show such limitations, given that transformers have full access to prior context through attention? We find that although a two-layer transformer can be trained to solve working memory tasks perfectly, a diverse set of pretrained LLMs continues to show working memory limitations. Notably, LLMs reproduce interference signatures observed in humans: performance degrades with increasing memory load and is biased by recency and stimulus statistics. Across models, stronger working memory capacity correlates with broader competence on standard benchmarks, mirroring its link to general intelligence in humans. Yet despite substantial variability in working memory performance, LLMs surprisingly converge on a common computational mechanism. Rather than directly copying the relevant memory item from context, models encode multiple memory items in entangled representations, such that successful recall depends on interference control -- actively suppressing task-irrelevant content to isolate the target for readout. Moreover, a targeted intervention that suppresses stimulus content information improves performance, providing causal support for representational interference. Together, these findings identify representational interference as a core constraint on working memory in pretrained LLMs, suggesting that working-memory limits in biological and artificial systems may reflect a shared computational challenge: selecting task-relevant information under interference.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2604.09670 [cs.LG] |
| (or arXiv:2604.09670v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.09670 arXiv-issued DOI via DataCite |
From: Hua-Dong Xiong [view email]
[v1]
Wed, 1 Apr 2026 17:19:46 UTC (1,582 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。