The cross covariogram g_{K,L} of two convex sets K and L in R^n is the function which associates to each x in R^n the volume of the intersection of K and L+x. Very recently Averkov and Bianchi [AB] have confirmed Matheron's conjecture on the covariogram problem, that asserts that any planar convex body K is determined by the knowledge of g_{K,K}. The problem of determining the sets from their covariogram is relevant in probability, in statistical shape recognition and in the determination of the atomic structure of a quasicrystal from X-ray diffraction images. We prove that when K and L are convex polygons (and also when K and L are planar convex cones) g_{K,L} determines both K and L, up to a described family of exceptions. These results imply that, when K and L are in these classes, the information provided by the cross covariogram is so rich as to determine not only one unknown body, as required by Matheron's conjecture, but two bodies, with a few classified exceptions. These results are also used by Bianchi [Bia] to prove that any convex polytope P in R^3 is determined by g_{P,P}.