This contribution aims at studying the behaviour of the classical sample moment estimator, $S(n,q)= \sum_{k=1}^n X_k^{q}/n $, as a function of the number of available samples $n$, in the case where the random variables $X$ are positive, have finite moments at all orders and are naturally of the form $X= \exp Y$ with the tail of $Y$ behaving like $e^{-y^ρ}$. This class of laws encompasses and generalizes the classical example of the log-normal law. This form is motivated by a number of applications stemming from modern statistical physics or multifractal analysis. Borrowing heuristic and analytical results from the analysis of the Random Energy Model in statistical physics, a critical moment $q_c(n)$ is defined as the largest statistical order $q$ up to which the sample mean estimator $S(n,q)$ correctly accounts for the ensemble average $\E X^q$, for a given $n$. A practical estimator for the critical moment $q_c(n)$ is then proposed. Its statistical performance are studied analytically and illustrated numerically in the case of \emph{i.i.d.} samples. A simple modification is proposed to explicitly account for correlation amongst the observed samples. Estimation performance are then carefully evaluated by means of Monte-Carlo simulations in the practical case of correlated time series.