For a subtorus $T \subseteq (\mathbb{R}/\mathbb{Z})^n$, let $D(T)$ denote the $L^\infty$-distance from $T$ to the point $(1/2, \ldots, 1/2)$. For a subtorus $U \subseteq (\mathbb{R}/\mathbb{Z})^n$, define $\mathcal{S}_1(U)$, the Lonely Runner spectrum relative to $U$, to be the set of all values of $D(T)$ as $T$ ranges over the $1$-dimensional subtori of $U$ not contained in the union of the coordinate hyperplanes of $(\mathbb{R}/\mathbb{Z})^n$. The relative spectrum $\mathcal{S}_1((\mathbb{R}/\mathbb{Z})^n)$ is the ordinary Lonely Runner spectrum that has been studied previously. Giri and the second author recently showed that the relative spectra $\mathcal{S}_1(U)$ for $2$-dimensional subtori $U \subseteq (\mathbb{R}/\mathbb{Z})^n$ essentially govern the accumulation points of the Lonely Runner spectrum $\mathcal{S}_1((\mathbb{R}/\mathbb{Z})^n)$. In the present work, we prove that such relative spectra $\mathcal{S}_1(U)$ have a very rigid arithmetic structure, and that one can explicitly find a complete characterization of each such relative spectrum with a finite calculation; carrying out this calculation for a few specific examples sheds light on previous constructions in the literature on the Lonely Runner Problem.