Permutons constructed from a Liouville quantum gravity surface and a pair of space-filling Schramm-Loewner evolutions (SLEs) have been shown -- or are conjectured -- to describe the scaling limit of various natural models of random constrained permutations. We prove that, in two distinct and natural settings, these permutons uniquely determine, modulo rotation, scaling, translation and reflection, both the Liouville quantum gravity surface and the pair of space-filling SLEs used in their construction. In other words, the Liouville quantum gravity surface and the pair of space-filling SLEs can be deterministically reconstructed from the permuton. Our results cover the cases of the skew Brownian permutons, the universal limits of pattern-avoiding permutations, and the meandric permuton, which is the conjectural permuton limit of permutations obtained from uniform meanders. In the course of the proof, we give a detailed description of how the support of the permuton relates to the multiple points of the two space-filling SLEs.