We consider a generalisation of a conjecture by Patterson and Wiedemann from 1983 on the Hamming distance of a function from $\mathbb{F}_q^n$ to $\mathbb{F}_q$ to the set of affine functions from $\mathbb{F}_q^n$ to $\mathbb{F}_q$. We prove the conjecture for each $q$ such that the characteristic of $\mathbb{F}_q$ lies in a subset of the primes with density $1$ and we prove the conjecture for all $q$ by assuming the generalised Riemann hypothesis. Roughly speaking, we show the existence of functions for which the distance to the affine functions is maximised when $n$ tends to infinity. This also determines the asymptotic behaviour of the covering radius of the $[q^n,n+1]$ Reed-Muller code over $\mathbb{F}_q$ and so answers a question raised by Leducq in 2013. Our results extend the case $q=2$, which was recently proved by the author and which corresponds to the original conjecture by Patterson and Wiedemann. Our proof combines evaluations of Gauss sums in the semiprimitive case, probabilistic arguments, and methods from discrepancy theory.