We introduce moduli spaces of colored graphs, defined as spaces of non-degenerate metrics on certain families of edge-colored graphs. Apart from fixing the rank and number of legs these families are determined by various conditions on the coloring of their graphs. The motivation for this is to study Feynman integrals in quantum field theory using the combinatorial structure of these moduli spaces. Here a family of graphs is specified by the allowed Feynman diagrams in a particular quantum field theory such as (massive) scalar fields or quantum electrodynamics. The resulting spaces are cell complexes with a rich and interesting combinatorial structure. We treat some examples in detail and discuss their topological properties, connectivity and homology groups.