We obtain tight lower bounds for the trace norm $\Vert \cdot \Vert_1$ of some matrices with diagonal zero, in terms of the entry-wise $L^1$-norm (denoted by $\Vert \cdot \Vert_{(1)}$). It is shown that on the space of nonzero real symmetric matrices $A$ of order $n$ with diagonal zero, the minimum value of the quantity $\frac{\Vert A\Vert_1}{\Vert A\Vert_{(1)}}$ is equal to $\frac{2}{n}$. The answer of the similar problem in the space of Hermitian matrices, is also obtained to be equal to $\tan(\fracπ{2n})$. The equivalent "dual" form of these results, give some upper bounds for the distance to the nearest diagonal matrix for a given symmetric or Hermitian matrix, when the distance is computed in the spectral norm.