Most of the security services in the connected world of cyber-physical systems necessitate authenticating a large number of nodes privately. In this paper, the private authentication problem is considered which consists of a certificate authority, a verifier (or some verifiers), many legitimate users (provers), and an arbitrary number of attackers. Each legitimate user wants to be authenticated (using his personal key) by the verifier(s), while simultaneously staying completely anonymous (even to the verifier). On the other hand, an attacker must fail to be authenticated. We analyze this problem from an information-theoretical perspective and propose a general interactive information-theoretic model for the problem. As a metric to measure the reliability, we consider the normalized total key rate whose maximization has a trade-off with establishing privacy. The problem is considered in two different scenarios: single-server scenario (only one verifier is considered, which all the provers are connected to) and multi-server scenario ($N$ verifiers are assumed, where each verifier is connected to a subset of users). For both scenarios, two regimes are considered: finite size regime (i.e., the variables are elements of a finite field) and asymptotic regime (i.e., the variables are considered to have large enough length). We propose achievable schemes that satisfy the completeness, soundness, and privacy properties in both single-server and multi-server scenarios in all cases. In the finite size regime, the main idea is to generate the authentication keys according to a secret sharing scheme. We show that the proposed scheme in the special case of multi-server authentication in the finite size regime is optimal. In the asymptotic regime, we use a random binning based scheme that relies on the joint typicality to generate the authentication keys.