We study the category whose objects are graphs of fixed genus and whose morphisms are contractions. We show that the corresponding contravariant module categories are Noetherian and we study two families of modules over these categories. The first takes a graph to a graded piece of the homology of its unordered configuration space and the second takes a graph to an intersection homology group whose dimension is given by a Kazhdan-Lusztig coefficient; in both cases we prove that the module is finitely generated. This allows us to draw conclusions about torsion in the homology groups of graph configuration spaces, and about the growth of Betti numbers of graph configuration spaces and Kazhdan-Lusztig coefficients of graphical matroids. We also explore the relationship between our category and outer space, which is used in the study of outer automorphisms of free groups.