Recently, the general problem of enumerating permutations $π=π_1\cdots π_n$ such that $π_{i+r}-π_i \neq s$ for all $1\leq i\leq n-r$, where $r$ and $s$ are fixed, was considered by Spahn and Zeilberger. In this paper, we consider an analogous problem on $k$-ary words involving the distribution of the corresponding statistic. Note that for $k$-ary words, it suffices to consider only the $r=1$ case of the aforementioned problem on permutations. Here, we compute for arbitrary $s$ an explicit formula for the ordinary generating function for $n \geq 0$ of the distribution of the statistic on $k$-ary words $ρ=ρ_1\cdotsρ_n$ recording the number of indices $i$ such that $ρ_{i+1}-ρ_i=s$. This result may then be used to find a comparable formula for finite set partitions with a fixed number of blocks, represented sequentially as restricted growth functions. Further, several sequences from the OEIS arise as enumerators of certain classes of $k$-ary words avoiding adjacencies with a prescribed difference. The comparable problem where one tracks indices $i$ such that the absolute difference $|a_{i+1}-a_i|$ is a fixed number is also considered on $k$-ary words and the corresponding generating function may be expressed in terms of Chebyshev polynomials. Finally, combinatorial proofs are found for several related recurrences and formulas for the total number of adjacencies of the form $a(a+s)$ on the various structures.