We introduce and study different compactifications of the moduli space of $n$ distinct weighted labeled points in a flag of affine spaces. We construct these spaces via the weighted and generalized Fulton-MacPherson compactifications of Routis and Kim-Sato. For certain weights, our compactifications are toric and isomorphic to the polypermutohedral and polystellahedral varieties, which arise in the work of Crowley-Huh-Larson-Simpson-Wang and Eur-Larson on polymatroids, a generalization of matroids. Moreover, we show that these toric compactifications have a fibration structure, with fibers isomorphic to the Losev-Manin space, and are related to each other via a geometric quotient.