Much effort has been spent on characterizing the spectrum of the non-backtracking matrix of certain classes of graphs, with special emphasis on the leading eigenvalue or the second eigenvector. Much less attention has been paid to the eigenvalues of small magnitude; here, we fully characterize the eigenvalues with magnitude equal to one. We relate the multiplicities of such eigenvalues to the existence of specific subgraphs. We formulate a conjecture on necessary and sufficient conditions for the diagonalizability of the non backtracking matrix. As an application, we establish an interlacing-type result for the Perron eigenvalue.