Let $\mathbb{P}_G([0,\infty))$ and $\mathbb{P}_G^{'}([0,\infty))$ be the sets of positive semidefinite and positive definite matrices of order $n$, respectively, with nonnegative entries, where some positions of zero entries are restricted by a simple graph $G$ with $n$ vertices. It is proved that for a connected simple graph $G$ of order $n\geq 3$, the set of powers preserving positive semidefiniteness on $\mathbb{P}_G([0,\infty))$ is precisely the same as the set of powers preserving positive definiteness on $\mathbb{P}_G^{'}([0,\infty))$. In particular, this provides an explicit combinatorial description of the critical exponent for positive definiteness, for all chordal graphs. Using chain sequences, it is proved that the Hadamard powers preserving the positive (semi) definiteness of every tridiagonal matrix with nonnegative entries are precisely $r\geq 1$. The infinite divisibility of tridiagonal matrices is studied. The same results are proved for a special family of pentadiagonal matrices.