The projective norm graphs $\text{NG}(q,t)$ provide tight constructions for the Turán number of complete bipartite graphs $K_{t,s}$ with $s>(t-1)!$. In this paper we determine their automorphism group and explore their small subgraphs. To this end we give quite precise estimates on the number of solutions of certain equation systems involving norms over finite fields. The determination of the largest integer $s_t$, such that the projective norm graph $\text{NG}(q,t)$ contains $K_{t,s_t}$ for all large enough prime powers $q$ is an important open question with far-reaching general consequences. The best known bounds, $t-1\leq s_t \leq (t-1)!$, are far apart for $t\geq 4$. Here we prove that $\text{NG}(q,4)$ does contain (many) $K_{4,6}$ for any prime power $q$ not divisble by $2$ or $3$. This greatly extends recent work of Grosu, using a completely different approach. Along the way we also count the copies of any fixed $3$-degenerate subgraph, and find that projective norm graphs are quasirandom with respect to this parameter. Some of these results also extend the work of Alon and Shikhelman on generalized Turán numbers. Finally we also give a new, more elementary proof for the $K_{4,7}$-freeness of $\text{NG}(q,4)$.