We investigate the statistical evidence for the use of `rough' fractional processes with Hurst exponent $H< 0.5$ for the modeling of volatility of financial assets, using a model-free approach. We introduce a non-parametric method for estimating the roughness of a function based on discrete sample, using the concept of normalized $p$-th variation along a sequence of partitions. We investigate the finite sample performance of our estimator for measuring the roughness of sample paths of stochastic processes using detailed numerical experiments based on sample paths of fractional Brownian motion and other fractional processes. We then apply this method to estimate the roughness of realized volatility signals based on high-frequency observations. Detailed numerical experiments based on stochastic volatility models show that, even when the instantaneous volatility has diffusive dynamics with the same roughness as Brownian motion, the realized volatility exhibits rough behaviour corresponding to a Hurst exponent significantly smaller than $0.5$. Comparison of roughness estimates for realized and instantaneous volatility in fractional volatility models with different values of Hurst exponent shows that, irrespective of the roughness of the spot volatility process, realized volatility always exhibits `rough' behaviour with an apparent Hurst index $\hat{H}<0.5$. These results suggest that the origin of the roughness observed in realized volatility time-series lies in the microstructure noise rather than the volatility process itself.