Characterizing the importances (i.e., centralities) of nodes in social, biological, and technological networks is a core topic in both network science and data science. We present a linear-algebraic framework that generalizes eigenvector-based centralities, including PageRank and hub/authority scores, to provide a common framework for two popular classes of multilayer networks: multiplex networks (which have layers that encode different types of relationships) and temporal networks (in which the relationships change over time). Our approach involves the study of joint, marginal, and conditional "supracentralities" that one can calculate from the dominant eigenvector of a supracentrality matrix [Taylor et al., 2017], which couples centrality matrices that are associated with individual network layers. We extend this prior work (which was restricted to temporal networks with layers that are coupled by adjacent-in-time coupling) by allowing the layers to be coupled through a (possibly asymmetric) interlayer-adjacency matrix $\tilde{\bf A}$, where the entry $\tilde{A}_{tt'} \geq 0$ encodes the coupling between layers $t$ and $t'$. Our framework provides a unifying foundation for centrality analysis of multiplex and temporal networks; it also illustrates a complicated dependency of the supracentralities on the topology and weights of interlayer coupling. By scaling $\tilde{\bf A}$ by an interlayer-coupling strength $ω\ge0$ and developing a singular perturbation theory for the limits of weak ($ω\to0^+$) and strong coupling ($ω\to\infty$), we also reveal an interesting dependence of supracentralities on the dominant left and right eigenvectors of $\tilde{\bf A}$.