Intelligent reflecting surfaces (IRSs) have been introduced into wireless communications systems due to their great potential to smartly customize and reconfigure radio propagation environments in a cost-effective manner. Despite the promising advantages of IRSs, academic research on IRSs is still in its infancy. In particular, the design and analysis of IRS-assisted wireless communication systems critically depend on an accurate and tractable modeling of the IRS. In this article, we first present and compare three IRS models, namely the conventional independent diffusive scatterer-based model, physics-based model, and impedance network-based model, in terms of their accuracy, tractability, and hardware complexity. Besides, a new framework based on partitioning the IRS into tiles and employing codebooks of transmission modes is introduced to enable scalable IRS optimization. Then, we investigate the impact of the three considered IRS models on system design, where several crucial technical challenges for the efficient design of IRS-assisted wireless systems are identified and the corresponding solutions are unraveled. Furthermore, to illustrate the properties of the considered models and the efficiency of the proposed solution concepts, IRS-assisted secure wireless systems and simultaneous wireless information and power transfer (SWIPT) systems are studied in more detail. Finally, several promising future research directions for IRS-assisted wireless systems are highlighted.