For $s \in {\mathbb{C}}$ and $0 < a <1$, let $ζ(s,a)$ and ${\rm{Li}}_s (e^{2πia})$ be the Hurwitz and periodic zeta functions, repectively. For $0 < a \le 1/2$, put $Z(s,a) := ζ(s,a) + ζ(s,1-a)$, $P(s,a) := {\rm{Li}}_s (e^{2πia}) + {\rm{Li}}_s (e^{2πi(1-a)})$, $Y(s,a) := ζ(s,a) - ζ(s,1-a)$ and $O(s,a):= -i \bigl( {\rm{Li}}_s (e^{2πia}) - {\rm{Li}}_s (e^{2πi(1-a)}) \bigr)$. Let $n \ge 0$ be an integer and $b := r/q$, where $q>r>0$ are coprime integers. In this paper, we prove that the values $Z(-n,b)$, $π^{-2n-2} P(2n+2,b)$, $Y(-n,b)$ and $π^{-2n-1} O(2n+1,b)$ are rational numbers, in addition, $π^{-2n-2} Z(2n+2,b)$, $P(-n,b)$, $π^{-2n-1} Y(2n+1,b)$ and $O(-n,b)$ are polynomials of $\cos (2π/q)$ and $\sin (2π/q)$ with rational coefficients. Furthermore, we show that $Z(-n,a)$, $π^{-2n-2} P(2n+2,a)$, $Y(-n,a)$ and $π^{-2n-1} O(2n+1,a)$ are polynomials of $0<a<1$ with rational coefficient, in addition, $π^{-2n-2} Z(2n+2,a)$, $P(-n,a)$, $π^{-2n-1} Y(2n+1,a)$ and $O(-n,a)$ are rational functions of $\exp (2 πia)$ with rational coefficients. Note that the rational numbers, polynomials and rational functions mentioned above are given explicitly. Moreover, we show that $P(s,a) \equiv 0$ for all $ 0 < a < 1/2$ if and only if $s$ is a negative even integer. We also prove similar assertions for $Z(s,a)$, $Y(s,a)$, $O(s,a)$ and so on. In addition, we prove that the function $Z(s,|a|)$ appears as the spectral density of some stationary self-similar Gaussian distributions.