Berenstein and Kazhdan's theory of geometric crystals gives rise to two commuting families of geometric crystal operators acting on the space of complex $m \times n$ matrices. These are birational actions, which we view as a crystal-theoretic analogue of the usual action of ${\rm SL}_m \times {\rm SL}_n$ on $m \times n$ matrices. We prove that the field of rational invariants (and ring of polynomial invariants) of each family of geometric crystal operators is generated by a set of algebraically independent polynomials, which are generalizations of the elementary symmetric polynomials in $m$ (or $n$) variables. We also give a set of algebraically independent generators for the intersection of these fields, and we explain how these fields are situated inside the larger fields of geometric $R$-matrix invariants, which were studied by Lam and the third-named author under the name loop symmetric functions. The key tool in our proof is the geometric RSK correspondence of Noumi and Yamada, which we show to be an isomorphism of geometric crystals. In an appendix jointly written with Thomas Lam, we prove the fundamental theorem of loop symmetric functions, which says that the polynomial invariants of the geometric $R$-matrix are generated by the loop elementary symmetric functions.