We study an approximate controllability problem for the continuity equation and its application to constructing transport maps with normalizing flows. Specifically, we construct time-dependent controls $θ=(w, a, b)$ in the vector field $x\mapsto w(a^\top x + b)_+$ to approximately transport a known base density $ρ_{\mathrm{B}}$ to a target density $ρ_*$. The approximation error is measured in relative entropy, and $θ$ are constructed piecewise constant, with bounds on the number of switches being provided. Our main result relies on an assumption on the relative tail decay of $ρ_*$ and $ρ_{\mathrm{B}}$, and provides hints on characterizing the reachable space of the continuity equation in relative entropy.