Acoustic propagation models are widely used in numerous oceanic and other underwater applications. Most conventional models are approximate solutions of the acoustic wave equation, and require accurate environmental knowledge to be available beforehand. Environmental parameters may not always be easily or accurately measurable. While data-driven techniques might allow us to model acoustic propagation without the need for extensive prior environmental knowledge, such techniques tend to be data-hungry and often infeasible in oceanic applications where data collection is difficult and expensive. We propose a data-aided ray physics based high frequency acoustic propagation modeling approach that enables us to train models with only a small amount of data. The proposed framework is not only data-efficient, but also offers flexibility to incorporate varying degrees of environmental knowledge, and generalizes well to permit extrapolation beyond the area where data was collected. We demonstrate the feasibility and applicability of our method through four numerical case studies, and one controlled experiment. We also benchmark our method's performance against classical data-driven techniques.