We consider Khintchine type inequalities on the $p$-th moments of vectors of $N$ pairwise independent Rademacher random variables. We establish that an analogue of Khintchine's inequality cannot hold in this setting with a constant that is independent of $N$; in fact, we prove that the best constant one can hope for is at least $N^{1/2-1/p}$. Furthermore, we show that this estimate is sharp for exchangeable vectors when $p = 4$. As a fortunate consequence of our work, we obtain similar results for $3$-wise independent vectors.