The arithmetic-geometric matrix $A_{ag}(G)$ of a graph $G$ is a square matrix, where the $(i,j)$-entry is equal to $\displaystyle \frac{d_{i}+d_{j}}{2\sqrt{d_{i}d_{j}}}$ if the vertices $v_{i}$ and $v_{j}$ are adjacent, and 0 otherwise. The arithmetic-geometric spectral radius of $G$, denoted by $ρ_{ag}(G)$, is the largest eigenvalue of the arithmetic-geometric matrix $A_{ag}(G)$. Let $S_{n}$ be the star of order $n\geq3$ and $S_{n}+e$ be the unicyclic graph obtained from $S_{n}$ by adding an edge. In this paper, we prove that for any tree $T$ of order $n\geq2$, $\displaystyle 2\cos\fracπ{n+1}\leqρ_{ag}(P_{n})\leqρ_{ag}(T)\leqρ_{ag}(S_{n})=\frac{n}{2},$ with equality if and only if $T\cong P_{n}$ for the lower bound, and if and only if $T\cong S_{n}$ for the upper bound. We also prove that for any unicyclic graph $G$ of order $n\geq3$, $\displaystyle 2=ρ_{ag}(C_{n})\leqρ_{ag}(G)\leqρ_{ag}(S_{n}+e),$ the lower (upper, respectively) bound is attained if and only if $T\cong C_{n}$ ($T\cong S_{n}+e$, respectively) and $\displaystyleρ_{ag}(S_{n}+e)<\frac{n}{2}$ for $n\geq7$.