Due to circuit failures, defective elements that cannot adaptively adjust the phase shifts of their impinging signals in a desired manner may exist on an intelligent reflecting surface (IRS). Traditional way to locate these defective IRS elements requires a thorough diagnosis of all the circuits belonging to a huge number of IRS elements, which is practically challenging. In this paper, we will devise novel approaches under which a transmitter sends known pilot signals and a receiver localizes all the defective IRS elements just based on its over-the-air measurements reflected from the IRS. Specifically, given any set of IRS elements, we propose an efficient method to process the received signals to determine whether this cluster contains defective elements or not with a very high accuracy probability. Based on this method, we show that the over-the-air diagnosis problem belongs to the 20 questions problem, where we can adaptively change the query set at the IRS so as to localize all the defective elements as quickly as possible. Along this line, we first propose a sorted posterior matching (sortPM) based method according to the noisy 20 questions technique, which enables accurate diagnosis even if the answers about the existence of defective elements in some sets of interest are wrong at certain question and answer (Q&A) rounds due to the noisy received signals. Next, to reduce the complexity, we propose a bisection based method according to the noiseless 20 questions technique, which totally trusts the answer at each Q&A round and keeps removing half of the remaining region based on such answers. Via numerical results, we show that our proposed methods can exploit the over-the-air measurements to localize all the defective IRS elements quickly and accurately.