We introduce a class of continuous Volterra processes, called Volterra clocks, and study their singular limit as the memory kernel collapses to a Dirac mass at zero. The dynamics are parametrised by a function $f$ acting as a nonlinear time-change, generalising the Volterra square-root process and recovering it when $f$ is affine. In the singular limit, the continuous Volterra clock converges weakly to a pure-jump process given by first passage times of a Brownian motion to curved boundaries, including affine and square-root boundaries when $f$ is, respectively, affine or quadratic. Outside the affine setting, characteristic function methods are no longer available, and we instead identify the limit directly from the dynamics. We do this through a topological framework adapted to the time-change structure which involves Skorokhod's $M_1$ topology and a decorated notion of convergence. Our analysis unifies several regimes of interest for general Volterra clocks, including large-time asymptotics, fast mean reversion, and hyper-roughness. In particular, this subsumes and extends existing results in the affine setting.