First-passage properties are central to the kinetics of target-search processes. Theoretical approaches so far primarily focused on predicting first-passage statistics for a given process or model. In practice, however, one faces the reverse problem of inferring first-passage statistics from, typically sub-sampled, experimental or simulation data. Obtaining trustworthy estimates from under-sampled data and unknown underlying dynamics remains a daunting task, and the assessment of the uncertainty is imperative. In this chapter, we highlight recent progress in understanding and controlling finite-sample effects in empirical first-passage times of reversible Markov processes. Precisely, we present concentration inequalities bounding from above the deviations of the sample mean for any sample size from the true mean first-passage time and construct non-asymptotic confidence intervals. Moreover, we present two-sided bounds on the range of fluctuations, i.e, deviations of the expected maximum and minimum from the mean in any given sample, which control uncertainty even in situations where the mean is a priori not a sufficient statistic.