Let $μ$ be a probability distribution on a multi-state spin system on a set $V$ of sites; equivalently, a $d$-partite simplicial complex with distribution $μ$ on maximal faces. For any pair of vertices $u,v\in V$, define the pairwise spectral influence $\mathcal{I}_{u,v}$ as follows. Let $σ$ be a choice of spins $s_w\in S_w$ for every $w\in V\setminus\{u,v\}$, and construct a matrix in $\mathbb{R}^{(S_u\cup S_v)\times (S_u\cup S_v)}$ where for any $s_u\in S_u, s_v\in S_v$, the $(us_u,vs_v)$-entry is the probability that $s_v$ is the spin of $v$ conditioned on $s_u$ being the spin of $u$ and on $σ$. Then $\mathcal{I}_{u,v}$ is the maximal second eigenvalue of this matrix, over all choices of spins for all $w\in V\setminus\{u,v\}$. Equivalently, $\mathcal{I}_{u,v}$ is the maximum local spectral expansion of links of codimension $2$ that include a spin for every $w \in V \setminus \{u,v\}$. We show that if the largest eigenvalue of the pairwise spectral influence matrix with entries $\mathcal{I}_{u,v}$ is bounded away from 1, i.e. $λ_{\max}(\mathcal{I})\leq 1-ε$ (and $X$ is connected), then the Glauber dynamics mixes rapidly and generate samples from $μ$. This improves/generalizes the classical Dobrushin's influence matrix as the $\mathcal{I}_{u,v}$ lower-bounds the classical influence of $u\to v$. As an application, we prove that the Glauber dynamics mixes rapidly up to (approximately) the phase transition for the multi-state hardcore model--a widely studied model in telecommunication networks and statistical physics (generalizing the hardcore model) introduced by Mazel and Suhov. As a by-product of our results, we also prove improved/almost optimal trickle-down theorems for partite simplicial complexes. Our proof builds on the trickle-down theorems via $\mathcal{C}$-Lorentzian polynomials machinery recently developed by the authors and Lindberg.