The packet routing problem asks to select routing paths that minimize the maximum edge congestion for a set of packets specified by source-destination vertex pairs. We revisit a semi-oblivious approach to this problem: each source-destination pair is assigned a small set of predefined paths before the demand is revealed, while the sending rates along the paths can be optimally adapted to the demand. This approach has been considered in practice in network traffic engineering due to its superior robustness and performance as compared to both oblivious routing and traditional traffic engineering approaches. We show the existence of sparse semi-oblivious routings: only $O(\log n)$ paths are selected between each pair of vertices. The routing is $(poly \log n)$-competitive for all demands against the offline-optimal congestion objective. Even for the well-studied case of hypercubes, no such result was known: our deterministic and oblivious selection of $O(\log n)$ paths is the first simple construction of a deterministic oblivious structure that near-optimally assigns source-destination pairs to few routes. Our results contrast the current solely-negative landscape of results for semi-oblivious routing. We give the sparsity-competitiveness trade-off for lower sparsities and nearly match it with a lower bound. Our construction is extremely simple: Sample the few paths from any competitive oblivious routing. Indeed, this natural construction was used in traffic engineering as an unproven heuristic. We give a satisfactory theoretical justification for their empirical effectiveness: the competitiveness of the construction improves exponentially with the number of paths. Finally, when combined with the recent hop-constrained oblivious routing, we also obtain sparse and competitive structures for the completion-time objective.