Millimeter-wave (mmWave) communication operated in frequency bands between 30 and 300 GHz has attracted extensive attention due to the potential ability of offering orders of magnitude greater bandwidths combined with further gains via beamforming and spatial multiplexing from multi-element antenna arrays. MmWave system may exploit the hybrid analog and digital precoding to achieve simultaneously the diversity, array and multiplexing gain with a lower cost of implementation. Motivated by this, in this paper, we investigate the design of hybrid precoder and combiner with sub-connected architecture, where each radio frequency chain is connected to only a subset of base station (BS) antennas from the perspective of energy efficient transmission. The problem of interest is a non-convex and NP-hard problem that is difficult to solve directly. In order to address it, we resort to design a two-layer optimization method to solve the problem of interest by exploiting jointly the interference alignment and fractional programming. First, the analog precoder and combiner are optimized via the alternating-direction optimization method (ADOM) where the phase shifter can be easily adjusted with an analytical structure. Then, we optimize the digital precoder and combiner based on an effective multiple-input multiple-output (MIMO) channel coefficient. The convergence of the proposed algorithms is proved using the monotonic boundary theorem and fractional programming theory. Extensive simulation results are given to validate the effectiveness of the presented method and to evaluate the energy efficiency performance under various system configurations.