In this paper, a novel approach for optimizing and managing resource allocation in wireless small cell networks (SCNs) with device-to-device (D2D) communication is proposed. The proposed approach allows to jointly exploit both the wireless and social context of wireless users for optimizing the overall allocation of resources and improving traffic offload in SCNs. This context-aware resource allocation problem is formulated as a matching game in which user equipments (UEs) and resource blocks (RBs) rank one another, based on utility functions that capture both wireless and social metrics. Due to social interrelations, this game is shown to belong to a class of matching games with peer effects. To solve this game, a novel, selforganizing algorithm is proposed, using which UEs and RBs can interact to decide on their desired allocation. The proposed algorithm is then proven to converge to a two-sided stable matching between UEs and RBs. The properties of the resulting stable outcome are then studied and assessed. Simulation results using real social data show that clustering of socially connected users allows to offload a substantially larger amount of traffic than the conventional context-unaware approach. These results show that exploiting social context has high practical relevance in saving resources on the wireless links and on the backhaul.