Heterogeneous network is a novel network architecture proposed in Long-Term-Evolution~(LTE), which highly increases the capacity and coverage compared with the conventional networks. However, in order to provide the best services, appropriate resource management must be applied. In this paper, we consider the joint optimization problem of user association, subchannel allocation, and power allocation for downlink transmission in Multi-cell Multi-association Orthogonal Frequency Division Multiple Access (OFDMA) heterogeneous networks. To solve the optimization problem, we first divide it into two subproblems: 1) user association and subchannel allocation for fixed power allocation; 2) power allocation for fixed user association and subchannel allocation. Subsequently, we obtain a locally optimal solution for the joint optimization problem by solving these two subproblems alternately. For the first subproblem, we derive the globally optimal solution based on graph theory. For the second subproblem, we obtain a Karush-Kuhn-Tucker (KKT) optimal solution by a low complexity algorithm based on the difference of two convex functions approximation (DCA) method. In addition, the multi-antenna receiver case and the proportional fairness case are also discussed. Simulation results demonstrate that the proposed algorithms can significantly enhance the overall network throughput.