For any lattice congruence of the weak order on permutations, N. Reading proved that gluing together the cones of the braid fan that belong to the same congruence class defines a complete fan, called a quotient fan, and V. Pilaud and F. Santos showed that it is the normal fan of a polytope, called a quotientope. In this paper, we provide a simpler approach to realize quotient fans based on Minkowski sums of elementary polytopes, called shard polytopes, which have remarkable combinatorial and geometric properties. In contrast to the original construction of quotientopes, this Minkowski sum approach extends to type $B$.