We investigate the spectrum of the non-backtracking matrix of a graph. In particular, we show how to obtain eigenvectors of the non-backtracking matrix in terms of eigenvectors of a smaller matrix. Furthermore, we find an expression for the eigenvalues of the non-backtracking matrix in terms of eigenvalues of the adjacency matrix and use this to upper-bound the spectral radius of the non-backtracking matrix and to give a lower bound on the spectrum. We also investigate properties of a graph that can be determined by the spectrum. Specifically, we prove that the number of components, the number of degree 1 vertices, and whether or not the graph is bipartite are all determined by the spectrum of the non-backtracking matrix.