Given a matroid or flag of matroids we introduce several broad classes of polynomials satisfying Deletion-Contraction identities, and study their singularities. There are three main families of polynomials captured by our approach: matroidal polynomials on a matroid (including matroid basis polynomials, configuration polynomials, Tutte polynomials); flag matroidal polynomials on a flag matroid; and Feynman integrands. The last class includes under general kinematics the inhomogeneous Feynman diagram polynomials which naturally arise in the Lee--Pomeransky form of the Feynman integral attached to a Feynman diagram. Assuming that the primary underlying matroid is connected and of positive rank (and in the flag case, has rank at least two), we show: a) in positive characteristic, homogeneous matroidal polynomials are strongly $F$-regular; b) over an algebraically closed field of characteristic zero, the associated jet schemes of (flag) matroidal polynomials as well as those of Feynman integrands are irreducible. Consequently, all these polynomials have rational singularities (or are smooth).