The Feistel Boomerang Connectivity Table and the related notion of $F$-Boomerang uniformity (also known as the second-order zero differential uniformity) has been recently introduced by Boukerrou et al.~\cite{Bouk}. These tools shall provide a major impetus in the analysis of the security of the Feistel network-based ciphers. In the same paper, a characterization of almost perfect nonlinear functions (APN) over fields of even characteristic in terms of second-order zero differential uniformity was also given. Here, we find a sufficient condition for an odd or even function over fields of odd characteristic to be an APN function, in terms of second-order zero differential uniformity. Moreover, we compute the second-order zero differential spectra of several APN or other low differential uniform functions, and show that our considered functions also have low second-order zero differential uniformity, though it may vary widely, unlike the case for even characteristic when it is always zero.