Abstract:Recent years have seen a surge in data-driven surrogates for dynamical systems that can be orders of magnitude faster than numerical solvers. However, many machine learning-based models such as neural operators exhibit spectral bias, attenuating high-frequency components that often encode small-scale structure. This limitation is particularly damaging in applications such as weather forecasting, where misrepresented high frequencies can induce long-horizon instability. To address this issue, we propose multi-scale wavelet transformers (MSWTs), which learn system dynamics in a tokenized wavelet domain. The wavelet transform explicitly separates low- and high-frequency content across scales. MSWTs leverage a wavelet-preserving downsampling scheme that retains high-frequency features and employ wavelet-based attention to capture dependencies across scales and frequency bands. Experiments on chaotic dynamical systems show substantial error reductions and improved long horizon spectral fidelity. On the ERA5 climate reanalysis, MSWTs further reduce climatological bias, demonstrating their effectiveness in a real-world forecasting setting.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2602.01486 [cs.LG] |
| (or arXiv:2602.01486v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.01486 arXiv-issued DOI via DataCite |
From: Xuesong Wang [view email]
[v1]
Sun, 1 Feb 2026 23:41:58 UTC (9,708 KB)
[v2]
Wed, 6 May 2026 03:00:29 UTC (9,726 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。