Considering a heterogeneous network (HetNet) system consisting of a macro tier overlaid with a second tier of small cells (SCs), this paper studies the mean square error (MSE) based precoding design to be employed by the macro base station and the SC nodes for multiple-input multiple-output (MIMO) downlinks. First, a new sum-MSE of all users based minimization problem is proposed aiming to design a set of macro cell (MC) and SC transmit precoding matrices or vectors. To solve it, two different algorithms are presented. One is via a relaxed-constraints based alternating optimization (RAO) realized by efficient alternating optimization and relaxing non-convex constraints to convex ones. The other is via an unconstrained alternating optimization with normalization (UAON) implemented by introducing the constraints into the iterations with the normalization operation. Second, a separate MSE minimization based precoding is proposed by considering the signal and interference terms corresponding to the macro tier and the SCs separately. Simulation results show that the sum-MSE based RAO algorithm provides the best MSE performance among the proposed schemes under a number of system configurations. When the number of antennas at the macro-BS is sufficiently large, the MSE of the separate MSE-based precoding is found to approach that of RAO and surpass that of UAON. Together, this paper provides a suite of three new precoding techniques that is expected to meet the need in a broad range of HetNet environments with adequate balance between performance and complexity.