We prove hypergraph variants of the celebrated Alon-Roichman theorem on spectral expansion of sparse random Cayley graphs. One of these variants implies that for every prime $p\geq 3$ and any $\varepsilon > 0$, there exists a set of directions $D\subseteq \mathbb{F}_p^n$ of size $O_{p,\varepsilon}(p^{(1-1/p +o(1))n})$ such that for every set $A\subseteq \mathbb{F}_p^n$ of density $α$, the fraction of lines in $A$ with direction in $D$ is within $\varepsilonα$ of the fraction of all lines in $A$. Our proof uses new deviation bounds for sums of independent random multi-linear forms taking values in a generalization of the Birkhoff polytope. The proof of our deviation bound is based on Dudley's integral inequality and a probabilistic construction of $\varepsilon$-nets. Using the polynomial method we prove that a Cayley hypergraph with edges generated by a set~$D$ as above requires $|D| \geq Ω_p(n^{p-1})$ for (our notion of) spectral expansion for hypergraphs.