In this note, we consider Szemerédi's theorem on $k$-term arithmetic progressions over finite fields $\mathbb{F}_p^n$, where the allowed set $S$ of common differences in these progressions is chosen randomly of fixed size. Combining a generalization of an argument of Altman with Moshkovitz--Zhu's bounds for the partition rank of a tensor in terms of its analytic rank, we (slightly) improve the best known lower bounds (due to Briët) on the size $|S|$ required for Szemerédi's theorem with difference in $S$ to hold asymptotically almost surely.