Many measurements on soft condensed matter (e.g., biological and materials) systems track low-dimensional observables projected from the full system phase space as a function of time. Examples are dynamic structure factors, spectroscopic and rheological response functions, and time series of distances derived from optical tweezers, single-molecule spectroscopy and molecular dynamics simulations. In many such systems the projection renders the reduced dynamics non-Markovian and the observable is not prepared in, or initially sampled from and averaged over, a stationary distribution. We prove that such systems always exhibit non-equilibrium, time asymmetric dynamics. That is, they evolve in time with a broken time-translation invariance in a manner closely resembling aging dynamics. We identify the entropy associated with the breaking of time-translation symmetry that is a measure of the instantaneous thermodynamic displacement of latent, hidden degrees of freedom from their stationary state. Dynamical time asymmetry is a general phenomenon, independent of the underlying energy surface, and is frequently even visible in measurements on systems that have fully reached equilibrium. This finding has fundamental implications for the interpretation of many experiments on, and simulations of, biological and materials systems.