Intelligent reflecting surface (IRS) is a cost-effective solution for achieving high spectrum and energy efficiency in future wireless networks by leveraging massive low-cost passive elements that are able to reflect the signals with adjustable phase shifts. Prior works on IRS mainly consider continuous phase shifts at reflecting elements, which are practically difficult to implement due to the hardware limitation. In contrast, we study in this paper an IRS-aided wireless network, where an IRS with only a finite number of phase shifts at each element is deployed to assist in the communication from a multi-antenna access point (AP) to multiple single-antenna users. We aim to minimize the transmit power at the AP by jointly optimizing the continuous transmit precoding at the AP and the discrete reflect phase shifts at the IRS, subject to a given set of minimum signal-to-interference-plus-noise ratio (SINR) constraints at the user receivers. The considered problem is shown to be a mixed-integer non-linear program (MINLP) and thus is difficult to solve in general. To tackle this problem, we first study the single-user case with one user assisted by the IRS and propose both optimal and suboptimal algorithms for solving it. Besides, we analytically show that as compared to the ideal case with continuous phase shifts, the IRS with discrete phase shifts achieves the same squared power gain in terms of asymptotically large number of reflecting elements, while a constant proportional power loss is incurred that depends only on the number of phase-shift levels. The proposed designs for the single-user case are also extended to the general setup with multiple users among which some are aided by the IRS. Simulation results verify our performance analysis as well as the effectiveness of our proposed designs as compared to various benchmark schemes.