We consider the problem of detecting a deformation from a symmetric Gaussian random $p$-tensor $(p\geq 3)$ with a rank-one spike sampled from the Rademacher prior. Recently in Lesieur et al. (2017), it was proved that there exists a critical threshold $β_p$ so that when the signal-to-noise ratio exceeds $β_p$, one can distinguish the spiked and unspiked tensors and weakly recover the prior via the minimal mean-square-error method. On the other side, Perry, Wein, and Bandeira (2017) proved that there exists a $β_p'<β_p$ such that any statistical hypothesis test can not distinguish these two tensors, in the sense that their total variation distance asymptotically vanishes, when the signa-to-noise ratio is less than $β_p'$. In this work, we show that $β_p$ is indeed the critical threshold that strictly separates the distinguishability and indistinguishability between the two tensors under the total variation distance. Our approach is based on a subtle analysis of the high temperature behavior of the pure $p$-spin model with Ising spin, arising initially from the field of spin glasses. In particular, we identify the signal-to-noise criticality $β_p$ as the critical temperature, distinguishing the high and low temperature behavior, of the Ising pure $p$-spin mean-field spin glass model.