We consider the problem of recovering a single or multiple frequency-sparse signals, which share the same frequency components, from a subset of regularly spaced samples. The problem is referred to as continuous compressed sensing (CCS) in which the frequencies can take any values in the normalized domain [0,1). In this paper, a link between CCS and low rank matrix completion (LRMC) is established based on an $\ell_0$-pseudo-norm-like formulation, and theoretical guarantees for exact recovery are analyzed. Practically efficient algorithms are proposed based on the link and convex and nonconvex relaxations, and validated via numerical simulations.