Accurate localization in non-line-of-sight (NLoS) environments remains challenging even with both angle-of-arrival (AoA) and time-of-arrival (ToA) measurements. In complex urban scenarios, the absence of line-of-sight (LoS) paths and the lack of environment prior knowledge make geometric based localization methods inapplicable, while prior-based approach such as fingerprinting is sensitive to environmental perturbations. This paper proposes a novel environment-aware localization framework enabled by the emerging concept called channel knowledge map (CKM). In the offline stage, AoA-ToA path signatures are learned by the CKM, with each path mapped to one candidate scatterer, thereby forming geometric priors within the environment. In the online stage, observed paths are matched to the CKM to extract high-confidence scatterers. Nonlinear least squares (NLS) method is then applied to jointly estimate the user and dominant scatterer locations. Even with imperfect CSI matching, geometric feasibility consistent with CKM scatterer priors provides corrective information and suppresses ambiguity. Simulations demonstrate that the proposed scheme outperforms fingerprinting and offers a robust and scalable solution to address the challenging NLoS localization for integrated sensing and communication (ISAC) systems.