In this paper, we design a new smart software-defined radio access network architecture which is flexible and traffic and density aware for the fifth generation (5G) of cellular wireless networks and beyond. The proposed architecture, based on network parameters such as density of users and system traffic, performs five important tasks namely, dynamic radio resource management (RRM), dynamic BS type selection, dynamic functionality splitting, dynamic transmission technology selection, and dynamic framing. In this regard, we first elaborate the structure of the proposed smart soft-RAN model and explain the details of the proposed architecture and RRM algorithms. Next, as a case study, based on the proposed architecture, we design a novel coordinated multi point beamforming technique to enhance the throughput of a virtualized software defined-based 5G network utilizing the combination of power domain non-orthogonal multiple access and multiple-input single-output downlink communication. In doing so, we formulate an optimization problem with the aim of maximizing the total throughput subject to minimum required data rate of each user and maximum transmit power constraint of each mobile virtual network operator and each BS, and find jointly the non-orthogonal set, beamforming, and subcarrier allocation. To solve the proposed optimization problem, based on the network density, we design two centralized and semi-centralized algorithms. Specifically, for the ultra-dense scenario, we use the centralized algorithm while the semi-centralized one is used for the high and moderate density scenarios. Numerical results illustrate the performance and signaling overhead of the proposed algorithms, e.g., taking computational limitations into account the number of supported users is increased by more than 60%.