The rate of convergence of simple random walk on the Heisenberg group over $Z/nZ$ with a standard generating set was determined by Bump et al [1,2]. We extend this result to random walks on the same groups with an arbitrary minimal symmetric generating set. We also determine the rate of convergence of simple random walk on higher-dimensional versions of the Heisenberg group with a standard generating set. We obtain our results via Fourier analysis, using an eigenvalue bound for sums of twisted circulant matrices. The key tool is a generalization of a version of the Heisenberg Uncertainty Principle due to Donoho-Stark [4].