Sylvester's criterion characterizes positive definite (PD) and positive semidefinite (PSD) matrices without the need of eigendecomposition. It states that a symmetric matrix is PD if and only if all of its leading principal minors are positive, and a symmetric matrix is PSD if and only if all of its principal minors are nonnegative. For an $m\times m$ symmetric matrix, Sylvester's criterion requires computing $m$ and $2^m-1$ determinants to verify it is PD and PSD, respectively. Therefore, it is less useful for PSD matrices due to the exponential growth in the number of principal submatrices as the matrix dimension increases. We provide a stronger Sylvester's criterion for PSD matrices which only requires to verify the nonnegativity of $m(m+1)/2$ determinants. Based on the new criterion, we provide a method to derive elementwise criteria for PD and PSD matrices. We illustrate the applications of our results in PD or PSD matrix completion and highlight their statistics applications via nonlinear semidefinite program.