The theta process is a stochastic process of number theoretical origin arising as a scaling limit of quadratic Weyl sums. It can be described in terms of the geodesic flow and an automorphic function on a homogeneous space. This process has several properties in common with Brownian motion such as its Hölder regularity, uncorrelated increments and quadratic variation. However, crucially, we show that the theta process is not a semimartingale, making Itô calculus techniques inapplicable. Instead, we use the celebrated rough paths theory to develop the stochastic calculus for the theta process. We do so by constructing the iterated integrals - the ``rough path" - above the theta process. Rough paths theory takes a signal and its iterated integrals and produces a vast and robust theory of stochastic differential equations. In addition, the rough path we construct can be described in terms of higher rank theta sums, via equidistribution of horocycle lifts.