In order to keep trace of information and grow up, the infant brain has to resolve the problem about where old information is located and how to index new ones. We propose that the immature prefrontal cortex (PFC) use its primary functionality of detecting hierarchical patterns in temporal signals as a second purpose to organize the spatial ordering of the cortical networks in the developing brain itself. Our hypothesis is that the PFC detects the hierarchical structure in temporal sequences in the form of ordinal patterns and use them to index information hierarchically in different parts of the brain. Henceforth, we propose that this mechanism for detecting patterns participates in the ordinal organization development of the brain itself; i.e., the bootstrapping of the connectome. By doing so, it gives the tools to the language-ready brain for manipulating abstract knowledge and planning temporally ordered information; i.e., the emergence of symbolic thinking and language. We will review neural models that can support such mechanisms and propose new ones. We will confront then our ideas with evidence from developmental, behavioral and brain results and make some hypotheses, for instance, on the construction of the mirror neuron system, on embodied cognition, and on the capacity of learning-to-learn.