Define a forward problem as $ρ_y = G_\#ρ_x$, where the probability distribution $ρ_x$ is mapped to another distribution $ρ_y$ using the forward operator $G$. In this work, we investigate the corresponding inverse problem: Given $ρ_y$, how to find $ρ_x$? Depending on whether $ G$ is overdetermined or underdetermined, the solution can have drastically different behavior. In the overdetermined case, we formulate a variational problem $\min_{ρ_x} D( G_\#ρ_x, ρ_y)$, and find that different choices of the metric $ D$ significantly affect the quality of the reconstruction. When $ D$ is set to be the Wasserstein distance, the reconstruction is the marginal distribution, while setting $ D$ to be a $φ$-divergence reconstructs the conditional distribution. In the underdetermined case, we formulate the constrained optimization $\min_{\{ G_\#ρ_x=ρ_y\}} E[ρ_x]$. The choice of $ E$ also significantly impacts the construction: setting $ E$ to be the entropy gives us the piecewise constant reconstruction, while setting $ E$ to be the second moment, we recover the classical least-norm solution. We also examine the formulation with regularization: $\min_{ρ_x} D( G_\#ρ_x, ρ_y) + α\mathsf R[ρ_x]$, and find that the entropy-entropy pair leads to a regularized solution that is defined in a piecewise manner, whereas the $W_2$-$W_2$ pair leads to a least-norm solution where $W_2$ is the 2-Wasserstein metric.