We show a simple local norm regularization algorithm that works with high probability. Namely, we prove that if the entries of a $n \times n$ matrix $A$ are i.i.d. symmetrically distributed and have finite second moment, it is enough to zero out a small fraction of the rows and columns of $A$ with largest $L_2$ norms in order to bring the operator norm of $A$ to the almost optimal order $O(\sqrt{\log \log n \cdot n})$. As a corollary, we also obtain a constructive procedure to find a small submatrix of $A$ that one can zero out to achieve the same goal. This work is a natural continuation of our recent work with R. Vershynin, where we have shown that the norm of $A$ can be reduced to the optimal order $O(\sqrt{n})$ by zeroing out just a small submatrix of $A$, but did not provide a constructive procedure to find this small submatrix. Our current approach extends the norm regularization techniques developed for the graph adjacency (Bernoulli) matrices in the works of Feige and Ofek, and Le, Levina and Vershynin to the considerably broader class of matrices.