We consider the function $x^{-1}$ that inverses a finite field element $x \in \mathbb{F}_{p^n}$ ($p$ is prime, $0^{-1} = 0$) and affine $\mathbb{F}_{p}$-subspaces of $\mathbb{F}_{p^n}$ such that their images are affine subspaces as well. It is proven that the image of an affine subspace $L$, $|L| > 2$, is an affine subspace if and only if $L = q \mathbb{F}_{p^k}$, where $q \in \mathbb{F}_{p^n}^{*}$ and $k \mid n$. In other words, it is either a subfield of $\mathbb{F}_{p^n}$ or a subspace consisting of all elements of a subfield multiplied by $q$. This generalizes the results that were obtained for linear invariant subspaces in 2006. As a consequence, we propose a sufficient condition providing that a function $A(x^{-1}) + b$ has no invariant affine subspaces $U$ of cardinality $2 < |U| < p^n$ for an invertible linear transformation $A: \mathbb{F}_{p^n} \to \mathbb{F}_{p^n}$ and $b \in \mathbb{F}_{p^n}^{*}$. As an example, it is shown that the condition works for S-box of AES. Also, we demonstrate that some functions of the form $αx^{-1} + b$ have no invariant affine subspaces except for $\mathbb{F}_{p^n}$, where $α, b \in \mathbb{F}_{p^n}^{*}$ and $n$ is arbitrary.