The speed of a class of graphs counts the number of graphs on the vertex set $\lbrace 1,\dots, n\rbrace$ inside the class as a function of $n$. In this paper, we investigate this function for many classes of graphs that naturally arise in discrete geometry, for example intersection graphs of segments or disks in the plane. While upper bounds follow from Warren's theorem (a variant of a theorem of Milnor and Thom), all the previously known lower bounds were obtained from ad hoc constructions for very specific classes. We prove a general theorem giving an essentially tight lower bound for the number of graphs on $\lbrace 1,\dots, n\rbrace$ whose edges are defined using the signs of a given finite list of polynomials, assuming these polynomials satisfy some reasonable conditions. This in particular implies lower bounds for the speed of many different classes of intersection graphs, which essentially match the known upper bounds. Our general result also gives essentially tight lower bounds for counting containment orders of various families of geometric objects, including circle orders and angle orders. Some of the applications presented in this paper are new, whereas others recover results of Alon-Scheinerman, Fox, McDiarmid-Müller and Shi. For the proof of our result we use some tools from algebraic geometry and differential topology.