Matrix recovery is raised in many areas. In this paper, we build up a framework for almost everywhere matrix recovery which means to recover almost all the $P\in {\mathcal M}\subset {\mathbb H}^{p\times q}$ from $Tr(A_jP), j=1,\ldots,N$ where $A_j\in V_j\subset {\mathbb H}^{p\times q}$. We mainly focus on the following question: how many measurements are needed to recover almost all the matrices in ${\mathcal M}$? For the case where both ${\mathcal M}$ and $V_j$ are algebraic varieties, we use the tools from algebraic geometry to study the question and present some results to address it under many different settings.