We present a finer quantitative version of an observation due to Breuillard, Green, Guralnick and Tao which tells that for finite non-bipartite Cayley graphs, once the nontrivial eigenvalues of their normalized adjacency matrices are uniformly bounded away from $1$, then they are also uniformly bounded away from $-1$. Unlike previous works which depend heavily on combinatorial arguments, we rely more on analysis of eigenfunctions. We establish a new explicit lower bound for the gap between $-1$ and the smallest normalized adjacency eigenvalue, which improves previous lower bounds in terms of edge-expansion, and is comparable to the best known lower bound in terms of vertex-expansion.