Ceballos and Pons generalized weak order on permutations to a partial order on certain labeled trees, thereby introducing a new class of lattices called $s$-weak order. They also generalized the Tamari lattice by defining a particular sublattice of $s$-weak order called the $s$-Tamari lattice. We prove that the homotopy type of each open interval in $s$-weak order and in the $s$-Tamari lattice is either a ball or sphere. We do this by giving $s$-weak order and the $s$-Tamari lattice a type of edge labeling known as an SB-labeling. We characterize which intervals are homotopy equivalent to spheres and which are homotopy equivalent to balls; we also determine the dimension of the spheres for the intervals yielding spheres.