The emergence of 6G wireless communication enables massive edge device access and supports real-time intelligent services such as the Internet of things (IoT) and vehicle-to-everything (V2X). However, the surge in edge devices connectivity renders wireless resource allocation (RA) tasks as large-scale constrained optimization problems, whereas the stringent real-time requirement poses significant computational challenge for traditional algorithms. To address the challenge, feasibility-aware learning-to-optimize (L2O) techniques have recently gained attention. These learning-based methods offer efficient alternatives to conventional solvers by directly learning mappings from system parameters to feasible and near-optimal solutions. This article provide a comprehensive review of L2O model designs and feasibility enforcement techniques and investigates the application of constrained L2O in wireless RA systems and. The paper also presents a case study to benchmark different L2O approaches in weighted sum rate problem, and concludes by identifying key challenges and future research directions.