We provide a graphical calculus for computing averages of tensor network diagrams with respect to the distribution of random vectors containing independent uniform complex phases. Our method exploits the order structure of the partially ordered set of uniform block permutations. A similar calculus is developed for random vectors consisting of independent uniform signs, based on the combinatorics of the partially ordered set of even partitions. We employ our method to extend some of the results by Johnston and MacLean on the family of local diagonal unitary invariant matrices. Furthermore, our graphical approach applies just as well to the real (orthogonal) case, where we introduce the notion of triplewise complete positivity to study the condition for separability of the relevant bipartite matrices. Finally, we analyze the twirling of linear maps between matrix algebras by independent diagonal unitary matrices, showcasing another application of our method.