We construct a continuous-time, positively divisible non-Markovian process with memory of the initial state that satisfies the differential Chapman--Kolmogorov equation. In the stationary state, the correlation function exhibits exponential decay, a behavior typically regarded as characteristic of Markovian dynamics. Nevertheless, the memory is preserved throughout the evolution of the process, manifesting itself in observable statistical quantities. We further demonstrate that mutual information serves as a reliable measure of the non-Markovian character of the process. As an application, we study a random walk driven by the constructed process and show that the memory effect breaks ergodicity and modifies transport properties such as the diffusion coefficient.