In this paper, we discuss the integer decomposition property for Cayley sums and Minkowski sums of lattice polytopes. In fact, we characterize when Cayley sums have the integer decomposition property in terms of Minkowski sums. Moreover, by using this characterization, we consider when Cayley sums and Minkowski sums of $2$-convex-normal lattice polytopes have the integer decomposition property. Finally, we also discuss the level property for Minkowski sums and Cayley sums.