Abstract:Time series forecasting is critical across multiple domains, where time series data exhibit both local patterns and global dependencies. While Transformer-based methods effectively capture global dependencies, they often overlook short-term local variations in time series. Recent methods that adapt large language models (LLMs) into time series forecasting inherit this limitation by treating LLMs as black-box encoders, relying solely on the final-layer output and underutilizing hierarchical representations. To address this limitation, we propose Logo-LLM, a novel LLM-based framework that explicitly extracts and models multi-scale temporal features from different layers of a pre-trained LLM. Through empirical analysis, we show that shallow layers of LLMs capture local dynamics in time series, while deeper layers encode global trends. Moreover, Logo-LLM introduces lightweight Local-Mixer and Global-Mixer modules to align and integrate features with the temporal input across layers. Extensive experiments demonstrate that Logo-LLM achieves superior performance across diverse benchmarks, with strong generalization in few-shot and zero-shot settings while maintaining low computational overhead.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2505.11017 [cs.LG] |
| (or arXiv:2505.11017v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2505.11017 arXiv-issued DOI via DataCite |
From: Wenjie Ou [view email]
[v1]
Fri, 16 May 2025 09:10:49 UTC (5,436 KB)
[v2]
Thu, 16 Apr 2026 11:43:59 UTC (8,069 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。