We show that a wide class of risk-constrained nonconvex functional optimization problems exhibit strong duality, regardless of nonconvexity. We develop two novel results under distinct sets of assumptions, establishing strong duality over both decomposable policy spaces (matching and extending prior work in the risk neutral case), and nondecomposable policy spaces with structure (e.g., continuity or smoothness), including certain universal finite-dimensional (fixed depth/width) neural network parametrizations as special cases (improving established results in the risk-neutral setting as well). We consider constraints featuring convex and positively homogeneous risk measures with bounded risk envelopes, generalizing expectations. Popular risk measures supported within our setting include the conditional value-at-risk (CVaR), the (even non-monotone) mean-absolute deviation (MAD), certain distributionally robust representations and more generally all real-valued coherent risk measures on the space $L_1$. We further discuss various generalizations of our base model, extensions for risk measures supported on $L_{p>1}$, implications in the context of mean-risk tradeoff models, as well as applications in wireless systems resource allocation, and supervised constrained learning. Our core proof technique appears to be new and relies on risk conjugate duality in tandem with J. J. Uhl's weak extension of A. A. Lyapunov's convexity theorem for vector measures taking values in infinite-dimensional Banach spaces.