In order to perform universal fault-tolerant quantum computation, one needs to implement a logical non-Clifford gate. Consequently, it is important to understand codes that implement such gates transversally. In this paper, we adopt an algebraic approach to characterize all stabilizer codes for which transversal $T$ and $T^{-1}$ gates preserve the codespace. Our Heisenberg perspective reduces this to a finite geometry problem that translates to the design of certain classical codes. We prove three corollaries: (a) For any non-degenerate $[[ n,k,d ]]$ stabilizer code supporting a physical transversal $T$, there exists an $[[ n,k,d ]]$ CSS code with the same property; (b) Triorthogonal codes are the most general CSS codes that realize logical transversal $T$ via physical transversal $T$; (c) Triorthogonality is necessary for physical transversal $T$ on a CSS code to realize the logical identity. The main tool we use is a recent efficient characterization of certain diagonal gates in the Clifford hierarchy (arXiv:1902.04022). We refer to these gates as Quadratic Form Diagonal (QFD) gates. Our framework generalizes all existing code constructions that realize logical gates via transversal $T$. We provide several examples and briefly discuss connections to decreasing monomial codes, pin codes, generalized triorthogonality and quasitransversality. We partially extend these results towards characterizing all stabilizer codes that support transversal $π/2^{\ell}$ $Z$-rotations. In particular, using Ax's theorem on residue weights of polynomials, we provide an alternate characterization of logical gates induced by transversal $π/2^{\ell}$ $Z$-rotations on a family of quantum Reed-Muller codes. We also briefly discuss a general approach to analyze QFD gates that might lead to a characterization of all stabilizer codes that support any given physical transversal $1$- or $2$-local diagonal gate.