We present new explicit upper bounds for the smoothness of the distribution of the random diagonal sum $S_n=\sum_{j=1}^nX_{j,π(j)}$ of a random $n\times n$ matrix $X=(X_{j,r})$, where the $X_{j,r}$ are independent integer valued random variables, and $π$ denotes a uniformly distributed random permutation on $\{1,\dots,n\}$ independent of $X$. As a measure of smoothness, we consider the total variation distance between the distributions of $S_n$ and $1+S_n$. Our approach uses a new auxiliary inequality for a generalized normalized matrix hafnian, which could be of independent interest. This approach is also used to prove upper bounds of the Lévy concentration function of $S_n$ in the case of independent real valued random variables $X_{j,r}$.