Let $\mathbb{F}_{p^m}$ be a finite field of cardinality $p^m$, where $p$ is a prime, and $k, N$ be any positive integers. We denote $R_k=F_{p^m}[u]/\langle u^k\rangle =F_{p^m}+uF_{p^m}+\ldots+u^{k-1}F_{p^m}$ ($u^k=0$) and $λ=a_0+a_1u+\ldots+a_{k-1}u^{k-1}$ where $a_0, a_1,\ldots, a_{k-1}\in F_{p^m}$ satisfying $a_0\neq 0$ and $a_1=1$. Let $r$ be a positive integer satisfying $p^{r-1}+1\leq k\leq p^r$. We defined a Gray map from $R_k$ to $F_{p^m}^{p^r}$ first, then prove that the Gray image of any linear $λ$-constacyclic code over $R_k$ of length $N$ is a distance invariant linear $a_0^{p^r}$-constacyclic code over $F_{p^m}$ of length $p^rN$. Furthermore, the generator polynomials for each linear $λ$-constacyclic code over $R_k$ of length $N$ and its Gray image are given respectively. Finally, some optimal constacyclic codes over $F_{3}$ and $F_{5}$ are constructed.