Reconfigurable intelligent surface (RIS) is promising for future 6G wireless communications. However, the increased number of RIS elements results in the high overhead for channel acquisition and the non-negligible power consumption. Therefore, how to improve the system capacity with limited RIS elements is essential. Unlike the classical regular RIS whose elements are arranged on a regular grid, in this paper, we propose an irregular RIS structure to improve the system capacity. The key idea is to irregularly configure a given number of RIS elements on an enlarged surface, which provides extra spatial degrees of freedom compared with the regular RIS. In this way, the received signal power can be enhanced, and thus the system capacity can be improved. Then, we formulate a joint topology and precoding optimization problem to maximize the capacity for irregular RIS-aided communication systems. Accordingly, a joint optimization algorithm with low complexity is proposed to alternately optimize the RIS topology and the precoding design. Particularly, a tabu search-based method is used to design the irregular RIS topology, and a neighbor extraction-based cross-entropy method is introduced to optimize the precoding design. Simulation results demonstrate that, subject to the constraint of limited RIS elements, the proposed irregular RIS can significantly enhance the system capacity.