A novel reconfigurable intelligent surface (RIS) aided non-orthogonal multiple access (NOMA) downlink transmission framework is proposed. We formulate a long-term stochastic optimization problem that involves a joint optimization of NOMA user partitioning and RIS phase shifting, aiming at maximizing the sum data rate of the mobile users (MUs) in NOMA downlink networks. To solve the challenging joint optimization problem, we invoke a modified object migration automation (MOMA) algorithm to partition the users into equal-size clusters. To optimize the RIS phase-shifting matrix, we propose a deep deterministic policy gradient (DDPG) algorithm to collaboratively control multiple reflecting elements (REs) of the RIS. Different from conventional training-then-testing processing, we consider a long-term self-adjusting learning model where the intelligent agent is capable of learning the optimal action for every given state through exploration and exploitation. Extensive numerical results demonstrate that: 1) The proposed RIS-aided NOMA downlink framework achieves an enhanced sum data rate compared with the conventional orthogonal multiple access (OMA) framework. 2) The proposed DDPG algorithm is capable of learning a dynamic resource allocation policy in a long-term manner. 3) The performance of the proposed RIS-aided NOMA framework can be improved by increasing the granularity of the RIS phase shifts. The numerical results also show that reducing the granularity of the RIS phase shifts and increasing the number of REs are two efficient methods to improve the sum data rate of the MUs.