Extending recent developments of Kra, Moreira, Richter and Roberson, we study infinite sumset patterns in $U^k(Φ)$-uniform subsets of the integers, defined via the local uniformity seminorms introduced by Host and Kra. We relate the degree $k$ of a $U^k(Φ)$-uniform set to the existence of a rich variety of sumset patterns. As a counterpart, we stablish higher order parity obstruction to sumsets arising from nilsystems. We also provide examples of $U^k(Φ)$-uniform sets for applications, including sets arising from the Thue-Morse and Rudin-Shapiro sequences.