Abstract:Electroencephalography (EEG) provides real-time insights into brain activity and supports diverse applications in neuroscience. While EEG foundation models (EFMs) have emerged to address the scalability issues of task-specific models, current approaches still yield clinically uninterpretable and weakly discriminative representations, inefficiently capturing global dependencies and neglecting important local neural events. We present CodeBrain, a two-stage EFM designed to fill this gap. In the first stage, we introduce the TFDual-Tokenizer, which decouples heterogeneous temporal and frequency EEG signals into discrete tokens, quadratically expanding the representation space to enhance discriminative power and offering domain-specific representation-level interpretability by suggesting potential links to neural events and spectral rhythms. In the second stage, we propose the multi-scale EEGSSM architecture, which combines structured global convolution with sliding window attention to efficiently capture both sparse long-range and local dependencies, reflecting the brain's small-world topology. Pretrained on the largest public EEG corpus, CodeBrain achieves strong generalization across eight downstream tasks and ten datasets under distribution shifts, supported by comprehensive ablations, scaling-law analyzes, and interpretability evaluations. The code and the pretrained weights are available at this https URL.
| Comments: | Published as a conference paper at the International Conference on Learning Representations (ICLR 2026) |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2506.09110 [cs.LG] |
| (or arXiv:2506.09110v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2506.09110 arXiv-issued DOI via DataCite |
From: Jingying Ma [view email]
[v1]
Tue, 10 Jun 2025 17:20:39 UTC (4,029 KB)
[v2]
Thu, 25 Sep 2025 14:55:31 UTC (7,741 KB)
[v3]
Wed, 29 Apr 2026 19:08:46 UTC (9,274 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。