In this work, we develop a Bayesian framework for solving inverse problems in which the unknown parameter belongs to a space of Radon measures taking values in a separable Hilbert space. The inherent ill-posedness of such problems is addressed by introducing suitable measure-valued priors that encode prior information and promote desired sparsity properties of the parameter. Under appropriate assumptions on the forward operator and noise model, we establish the well-posedness of the Bayesian formulation by proving the existence, uniqueness, and stability of the posterior with respect to perturbations in the observed data. In addition, we also discuss computational strategies for approximating the posterior distribution. Finally, we present some examples that demonstrate the effectiveness of the proposed approach.