In an earlier paper, two of the authors defined a $5$-vertex Yang-Baxter algebra (a Hopf algebra) which acts on the sum of the equivariant quantum K-rings of Grassmannians $\mathrm{Gr}(k;n)$, where $k$ varies from $0$ to $n$. We construct geometrically defined operators on quantum K-rings describing this action. In particular, the $R$-matrix defining the Yang-Baxter algebra corresponds to the left Weyl group action. Most importantly, we use the `quantum=classical' statement for the quantum K-theory of Grassmannians to prove an explicit geometric interpretation of the action of generators of the Yang-Baxter algebra. The diagonal entries of the monodromy matrix are given by quantum K-multiplications by explicitly defined classes, and the off-diagonal entries by certain push-pull convolutions. We use this to find a quantization of the classes of fixed points in the quantum K-rings, corresponding to the Bethe vectors of the Yang-Baxter algebra. On each of the quantum K-rings, we prove that the two Frobenius structures (one from geometry, and the other from the integrable system construction) coincide. We discuss several applications, including an action of the extended affine Weyl group on the quantum K-theory ring (extending the Seidel action), a quantum version of the localization map (which is a ring homomorphism with respect to the quantum K-product), and a graphical calculus to multiply by Hirzebruch $λ_y$ classes of the dual of the tautological quotient bundle. In an Appendix we illustrate our results in the case when $n=2$.