Geometric quantiles are popular location functionals to build rank-based statistical procedures in multivariate settings. They are obtained through the minimization of a non-smooth convex objective function. As a result, the singularity of the directional derivatives leads to numerical instabilities and poor sample properties as well as surprising `phase transitions' from empirical to population distributions. To solve these issues, we introduce a regularized version of geometric distribution functions and quantiles that are provably close to the usual geometric concepts and share their qualitative properties, both in the empirical and continuous case, while allowing for a much broader applicability of asymptotic results without any moment condition. We also show that any linear assignment of probability measures (such as the univariate distribution function), that is also translation- and orthogonal-equivariant, necessarily coincides with one of our regularized geometric distribution functions.