For any integer $k>2$, the infinite $k$-bonacci word $W^{(k)}$, on the infinite alphabet is defined as the fixed point of the morphism $\varphi_k:\mathbb{N}\rightarrow \mathbb{N}^2 \cup \mathbb{N}$, where \begin{equation*} \varphi_k(ki+j) = \left\{ \begin{array}{ll} (ki)(ki+j+1) & \text{if } j = 0,\cdots ,k-2, (ki+j+1)& \text{if } j =k-1. \end{array} \right. \end{equation*} The finite $k$-bonacci word $W^{(k)}_n$ is then defined as the prefix of $W^{(k)}$ whose length is the $(n+k)$-th $k$-bonacci number. We obtain the structure of all square factors occurring in $W^{(k)}$. Moreover, we prove that the critical exponent of $W^{(k)}$ is $3-\frac{3}{2^k-1}$. Finally, we provide all critical factors of $W^{(k)}$.