Codebooks with small inner-product correlation are applied in many practical applications including direct spread code division multiple access (CDMA) communications, space-time codes and compressed sensing. It is extremely difficult to construct codebooks achieving the Welch bound or the Levenshtein bound. Constructing nearly optimal codebooks such that the ratio of its maximum cross-correlation amplitude to the corresponding bound approaches 1 is also an interesting research topic. In this paper, we firstly study a family of interesting character sums called generalized Jacobi sums over finite fields. Then we apply the generalized Jacobi sums and their related character sums to obtain two infinite classes of nearly optimal codebooks with respect to the Welch or Levenshtein bound. The codebooks can be viewed as generalizations of some known ones and contain new ones with very flexible parameters.