Permutation polynomials over finite fields are fundamental objects as they are used in various theoretical and practical applications in cryptography, coding theory, combinatorial design, and related topics. This family of polynomials constitutes an active research area in which advances are being made constantly. In particular, constructing infinite classes of permutation polynomials over finite fields with good differential properties (namely, low) remains an exciting problem despite much research in this direction for many years. This article exhibits low differentially uniform power permutations over finite fields of odd characteristic. Specifically, its objective is twofold concerning the power functions $F(x)=x^{\frac{p^n+3}{2}}$ defined over the finite field $F_{p^n}$ of order $p^n$, where $p$ is an odd prime, and $n$ is a positive integer. The first is to complement some former results initiated by Helleseth and Sandberg in \cite{HS} by solving the open problem left open for more than twenty years concerning the determination of the differential spectrum of $F$ when $p^n\equiv3\pmod 4$ and $p\neq 3$. The second is to determine the exact value of its differential uniformity. Our achievements are obtained firstly by evaluating some exponential sums over $F_{p^n}$ (which amounts to evaluating the number of $F_{p^n}$-rational points on some related curves and secondly by computing the number of solutions in $(F_{p^n})^4$ of a system of equations presented by Helleseth, Rong, and Sandberg in ["New families of almost perfect nonlinear power mappings," IEEE Trans. Inform. Theory, vol. 45. no. 2, 1999], naturally appears while determining the differential spectrum of $F$. We show that in the considered case ($p^n\equiv3\pmod 4$ and $p\neq 3$), $F$ is an APN power permutation when $p^n=11$, and a differentially $4$-uniform power permutation otherwise.