For spin systems, such as the $q$-colorings and independent-set models, approximating the partition function in the so-called non-uniqueness region, where the model exhibits long-range correlations, is typically computationally hard for bounded-degree graphs. We present new algorithmic results for approximating the partition function and sampling from the Gibbs distribution for spin systems in the non-uniqueness region on random regular bipartite graphs. We give an $\mathsf{FPRAS}$ for counting $q$-colorings for even $q=O\big(\tfracΔ{\logΔ}\big)$ on almost every $Δ$-regular bipartite graph. This is within a factor $O(\logΔ)$ of the sampling algorithm for general graphs in the uniqueness region and improves significantly upon the previous best bound of $q=O\big(\tfrac{\sqrtΔ}{(\logΔ)^2}\big)$ by Jenssen, Keevash, and Perkins (SODA'19). Analogously, for the hard-core model on independent sets weighted by $λ>0$, we present an $\mathsf{FPRAS}$ for estimating the partition function when $λ=Ω\big(\tfrac{\logΔ}Δ\big)$, which improves upon previous results by an $Ω(\log Δ)$ factor. Our results for the colorings and hard-core models follow from a general result that applies to arbitrary spin systems. Our main contribution is to show how to elevate probabilistic/analytic bounds on the marginal probabilities for the typical structure of phases on random bipartite regular graphs into efficient algorithms, using the polymer method. We further show evidence that our result for colorings is within a constant factor of best possible using current polymer-method approaches.