Let $V$ denote an $r$-dimensional $\mathbb{F}_{q^n}$-vector space. For an $m$-dimensional $\mathbb{F}_q$-subspace $U$ of $V$ assume that $\dim_q \left(\langle {\bf v}\rangle_{\mathbb{F}_{q^n}} \cap U\right) \geq 2$ for each non zero vector ${\bf v}\in U$. If $n\leq q$ then we prove the existence of an integer $1<d \mid n$ such that the set of one-dimensional $\mathbb{F}_{q^n}$-subspaces generated by non-zero vectors of $U$ is the same as the set of one-dimensional $\mathbb{F}_{q^n}$-subspaces generated by non-zero vectors of $\langle U\rangle_{\mathbb{F}_{q^d}}$. If we view $U$ as a point set of $\mathrm{AG}(r,q^n)$, it means that $U$ and $\langle U \rangle_{\mathbb{F}_{q^d}}$ determine the same set of directions. We prove a stronger statement when $n \mid m$. In terms of linear sets it means that an $\mathbb{F}_q$-linear set of $\mathrm{PG}(r-1,q^n)$ has maximum field of linearity $\mathbb{F}_q$ only if it has a point of weight one. We also present some consequences regarding the size of a linear set.