High-dimensional vector embeddings are widely used in retrieval systems, but they often suffer from noise, the curse of dimensionality, and slow runtime. However, dimensionality reduction (DR) is rarely applied due to its tendency to distort the nearest-neighbor (NN) structure that is critical for search. Existing DR techniques such as PCA and UMAP optimize global or manifold-preserving criteria, rather than retrieval-specific objectives. We present QPAD -- Quantile-Preserving Approximate Dimension Reduction, an unsupervised DR method that explicitly preserves approximate NN relations by maximizing the margin between k-NNs and non-k-NNs under a soft orthogonality constraint. We analyze its complexity and favorable properties. This design enables QPAD to retain ANN-relevant geometry without supervision or changes to the original embedding model, while supporting scalability for large-scale vector search and being indexable for ANN search. Experiments across five domains show that QPAD consistently outperforms eleven standard DR methods in preserving neighborhood structure, enabling more accurate search in reduced dimensions.