Many results on the convex order in the literature were stated for random variables with finite mean. For instance, a fundamental result in dependence modeling is that the sum of a pair of random random variables is upper bounded in convex order by that of its comonotonic version and lower bounded by that of its counter-monotonic version, and all existing proofs of this result require the random variables' expectations to be finite. We show that the above result remains true even when discarding the finite-mean assumption, and obtain several other results on the comparison of infinite-mean random variables via the convex order. To our surprise, we find two deceivingly similar definitions of the convex order, both of which exist widely in the literature, and they are not equivalent for random variables with infinite mean. This subtle discrepancy in definitions also applies to the supermodular order, and it gives rise to some incorrect statements, often found in the literature.