The deployment of small cells, overlaid on existing cellular infrastructure, is seen as a key feature in next-generation cellular systems. In this paper, the problem of user association in the downlink of small cell networks (SCNs) is considered. The problem is formulated as a many-to-one matching game in which the users and SCBSs rank one another based on utility functions that account for both the achievable performance, in terms of rate and fairness to cell edge users, as captured by newly proposed priorities. To solve this game, a novel distributed algorithm that can reach a stable matching is proposed. Simulation results show that the proposed approach yields an average utility gain of up to 65% compared to a common association algorithm that is based on received signal strength. Compared to the classical deferred acceptance algorithm, the results also show a 40% utility gain and a more fair utility distribution among the users.