Trajectory data, including time series and longitudinal measurements, are increasingly common in health-related domains such as biomedical research and epidemiology. Real-world trajectory data frequently exhibit heterogeneity across subjects such as patients, sites, and subpopulations, yet many traditional methods are not designed to accommodate such heterogeneity in data analysis. To address this, we propose a unified framework, termed Functional Singular Value Decomposition (FSVD), for statistical learning with heterogeneous trajectories. We establish the theoretical foundations of FSVD and develop a corresponding estimation algorithm that accommodates noisy and irregular observations. We further adapt FSVD to a wide range of trajectory-learning tasks, including dimension reduction, factor modeling, regression, clustering, and data completion, while preserving its ability to account for heterogeneity, leverage inherent smoothness, and handle irregular sampling. Through extensive simulations, we demonstrate that FSVD-based methods consistently outperform existing approaches across these tasks. Finally, we apply FSVD to a COVID-19 case-count dataset and electronic health record datasets, showcasing its effective performance in global and subgroup pattern discovery and factor analysis.