This paper studies one-sided hypothesis testing under random sampling without replacement. That is, when $n+1$ binary random variables $X_1,\ldots, X_{n+1}$ are subject to a permutation invariant distribution and $n$ binary random variables $X_1,\ldots, X_{n}$ are observed, we have proposed randomized tests with a randomization parameter for the upper confidence limit of the expectation of the $(n+1)$th random variable $X_{n+1}$ under a given significance level $δ>0$. Our proposed randomized test significantly improves over deterministic test unlike random sampling with replacement.