Let $P_0$ be a probability on the real line generating a natural exponential family $(P_t)_{t\in \mathbb {R}}$. Fix $α$ in $ (0,1).$ We show that the property that $P_t((-\infty,t)) \leq α\leq P_t((-\infty,t])$ for all $t$ implies that there exists a number $μ_α$ such that $P_0$ is the Gaussian distribution $N(μ_α,1).$ In other terms, if for all $t$, $t$ is a quantile of $P_t$ associated to some threshold $α\in (0,1)$, then the exponential family must be Gaussian. The case $α=1/2$, \textit{i.e.} $t$ is always a median of $P_t,$ has been considered in Letac \textit{et al.} (2018). Analogously let $Q$ be a measure on $[0,\infty)$ generating a natural exponential family $(Q_{-t})_{t>0}$. We show that $Q_{-t}([0,t^{-1}))\leq α\leq Q_{-t}([0,t^{-1}])$ for all $t>0$ implies that there exists a number $p=p_α>0$ such that $Q(dx)\propto x^{p-1}dx,$ and thus $Q_{-t}$ has to be a gamma distribution with parameters $p$ and $t.$