Consider the population model with infinite size associated to subcritical continuous-state branching processes (CSBP). Individuals reproduce independently according to the same subcritical offspring distribution. We study the long-term behaviour of the ancestral lineages as time goes to the past and show that the flow of ancestral lineages, properly renormalized, converges almost surely to the inverse of a drift-free subordinator whose Laplace exponent is explicit in terms of the branching mechanism. We provide an interpretation in terms of the genealogy of the population. In particular, we show that the inverse subordinator is partitioning the current population into ancestral families with distinct common ancestors. When Grey's condition is satisfied, the population comes from a discrete set of ancestors and the ancestral families are i.i.d and distributed according to the quasi-stationary distribution of the CSBP conditioned on non-extinction. When Grey's condition is not satisfied, the population comes from a continuum of ancestors which is described as the set of increase points $\mathscr{S}$ of the limiting inverse subordinator. The Hausdorff dimension of $\mathscr{S}$ is given. The proof is based on a general result for stochastically monotone processes of independent interest, which relates $θ$-invariant measures and $θ$-invariant functions for a process and its Siegmund dual.