A level graph is the data of a pair $(G,π)$ consisting of a finite graph $G$ and an ordered partition $π$ on the set of vertices of $G$. To each level graph on $n$ vertices we associate a polytope in $\mathbb R^n$ called its residue polytope. We show that residue polytopes are compatible with each other in the sense that if $π'$ is a coarsening of $π$, then the polytope associated to $(G,π)$ is a face of the one associated to $(G,π')$. Moreover, they form all the faces of the residue polytope of $G$, defined as the polytope associated to the level graph with the trivial ordered partition. The results are used in a companion work to describe limits of spaces of Abelian differentials on families of Riemann surfaces approaching a stable Riemann surface on the boundary of the moduli space.