Many multivariate data analysis techniques for an $m\times n$ matrix $\m Y$ are related to the model $\m Y = \m M +\m E$, where $\m Y$ is an $m\times n$ matrix of full rank and $\m M$ is an unobserved mean matrix of rank $K< (m\wedge n)$. Typically the rank of $\m M$ is estimated in a heuristic way and then the least-squares estimate of $\m M$ is obtained via the singular value decomposition of $\m Y$, yielding an estimate that can have a very high variance. In this paper we suggest a model-based alternative to the above approach by providing prior distributions and posterior estimation for the rank of $\m M$ and the components of its singular value decomposition. In addition to providing more accurate inference, such an approach has the advantage of being extendable to more general data-analysis situations, such as inference in the presence of missing data and estimation in a generalized linear modeling framework.