Finding functions, particularly permutations, with good differential properties has received a lot of attention due to their varied applications. For instance, in combinatorial design theory, a correspondence of perfect $c$-nonlinear functions and difference sets in some quasigroups was recently shown by Anbar et al. (J. Comb. Des. 31(12):1-24, 2023). Additionally, in a recent manuscript by Pal et al. (Adv. Math. Communications, to appear), a very interesting connection between the $c$-differential uniformity and boomerang uniformity, when $c=-1$, was pointed out, showing that they are the same for an odd APN permutation, sparking yet more interest in the construction of functions with low $c$-differential uniformity. We investigate the $c$-differential uniformity of some classes of permutation polynomials. As a result, we add four more classes of permutation polynomials to the family of functions that only contains a few (non-trivial) perfect $c$-nonlinear functions over finite fields of even characteristic. Moreover, we include a class of permutation polynomials with low $c$-differential uniformity over the field of characteristic~$3$. To solve the involved equations over finite fields, we use various number theoretical techniques, in particular, we find explicitly many Walsh transform coefficients and Weil sums that may be of an independent interest.