We study time-inhomogeneous random walks on finite groups in the case where each random walk step need not be supported on a generating set of the group. When the supports of the random walk steps satisfy a natural condition involving normal subgroups of quotients of the group, we show that the random walk converges to the uniform distribution on the group and give bounds for the convergence rate using spectral properties of the random walk steps. As an application, we use the moment method of Wood to prove a universality theorem for cokernels of random integer matrices allowing some dependence between entries.