Let $\mathcal{G}=\{G_1, G_2, \ldots , G_k\}$ be a family of bipartite graphs on the same vertex set. A rainbow Hamilton path (cycle) in $\mathcal{G}$ is a path (cycle) that visits each vertex precisely once such that any two edges belong to different graphs of $\mathcal{G}.$ In this paper, by adopting the technique of bi-shifting, we present tight sufficient conditions in terms of the spectral radius for a family $\mathcal{G}$ to admit a rainbow Hamilton path and cycle, respectively. Meanwhile, we completely characterize the corresponding spectral extremal graphs.