This work examines the performance gain achieved by deploying an intelligent reflecting surface (IRS) in covert communications. To this end, we formulate the joint design of the transmit power and the IRS reflection coefficients by taking into account the communication covertness for the cases with global channel state information (CSI) and without a warden's instantaneous CSI. For the case of global CSI, we first prove that perfect covertness is achievable with the aid of the IRS even for a single-antenna transmitter, which is impossible without an IRS. Then, we develop a penalty successive convex approximation (PSCA) algorithm to tackle the design problem. Considering the high complexity of the PSCA algorithm, we further propose a low-complexity two-stage algorithm, where analytical expressions for the transmit power and the IRS's reflection coefficients are derived. For the case without the warden's instantaneous CSI, we first derive the covertness constraint analytically facilitating the optimal phase shift design. Then, we consider three hardware-related constraints on the IRS's reflection amplitudes and determine their optimal designs together with the optimal transmit power. Our examination shows that significant performance gain can be achieved by deploying an IRS into covert communications.