The \emph{packing chromatic number $χ_ρ(G)$} of a graph $G$ is the smallest integer $k$ for which there exists a vertex coloring $Γ: V(G)\rightarrow \{1,2,\dots , k\}$ such that any two vertices of color $i$ are at distance at least $i + 1$. For $g\in \{6,8,12\}$, $(q+1,g)$-Moore graphs are $(q+1)$-regular graphs with girth $g$ which are the incidence graphs of a symmetric generalized $g/2$-gons of order $q$. In this paper we study the packing chromatic number of a $(q+1,g)$-Moore graph $G$. For $g=6$ we present the exact value of $χ_ρ(G)$. For $g=8$, we determine $χ_ρ(G)$ in terms of the intersection of certain structures in generalized quadrangles. For $g=12$, we present lower and upper bounds for this invariant when $q\ge 9$ an odd prime power.