The symplectic geometry mode decomposition (SGMD) is a powerful method for analyzing time sequences. The SGMD is based on the upper conversion via embedding and down conversion via diagonal averaging principle (DAP) inherited from the singular spectrum analysis (SSA). However, there are two defects in the DAP: it just hold for the time delay $τ=1$ in the trajectory matrix and it fails for the time sequence of type-1 with the form $X=\{x[n]\}^N_{n=1}$. In order to overcome these disadvantages, the inverse step for embedding is explored with binary Diophantine equation in number theory. The contributions of this work lie in three aspects: firstly, the pulling back theorem is proposed and proved, which state the general formula for converting the component of trajectory matrix to the component of time sequence for the general representation of time sequence and for any time delay $τ\ge 1$; secondly a unified framework for decomposing both the deterministic and random time sequences into multiple modes is presented and explained; finally, the guidance of configuring the time delay is suggested, namely the time delay should be selected in a limited range via balancing the efficiency of matrix computation and accuracy of state estimation. It could be expected that the pulling back theorem will help the researchers and engineers to deepen the understanding of the theory and extend the applications of the SGMD and SSA in analyzing time sequences.