We establish formulas for the Hilbert series of the Chow ring of a polymatroid using arbitrary building sets. For braid matroids and minimal building sets, our results produce new formulas for the Poincaré polynomial of the moduli space $\overline{\mathcal{M}}_{0,n+1}$ of pointed stable rational curves, and recover several previous results by Keel, Getzler, Manin, and Aluffi--Marcolli--Nascimento. We also use our methods to produce examples of matroids and building sets for which the corresponding Chow ring has Hilbert series with non-log-concave coefficients. This contrasts with the real-rootedness and log-concavity conjectures of Ferroni--Schröter for matroids with maximal building sets, and of Aluffi--Chen--Marcolli for braid matroids with minimal building sets.