Let $cφ_{k}(n)$ be the number of $k$-colored generalized Frobenius partitions of $n$. We establish some infinite families of congruences for $cφ_{3}(n)$ and $cφ_{9}(n)$ modulo arbitrary powers of 3, which refine the results of Kolitsch. For example, for $k\ge 3$ and $n\ge 0$, we prove that \[cφ_{3}\Big(3^{2k}n+\frac{7\cdot 3^{2k}+1}{8}\Big) \equiv 0 \pmod{3^{4k+5}}.\] We give two different proofs to the congruences satisfied by $cφ_{9}(n)$. One of the proofs uses an relation between $cφ_{9}(n)$ and $cφ_{3}(n)$ due to Kolitsch, for which we provide a new proof in this paper.