This paper introduces a novel framework that synergistically combines Non-Orthogonal Multiple Access (NOMA) with Color Shift Keying (CSK) modulation to substantially boost spectral efficiency in Visible Light Communication (VLC) systems. A key challenge in the proposed NOMA-CSK architecture is managing the complex power allocation process, especially under cross-color interference caused by spectral overlap among LEDs and the limitations of optical filters. To overcome this, we develop an intelligent power allocation strategy powered by a Soft Actor-Critic (SAC) reinforcement learning agent. Trained in a simulated indoor environment, the SAC agent dynamically distributes power among users with diverse channel conditions while balancing multiple performance objectives. Simulation results show that our SAC-based method significantly outperforms traditional approaches such as Gain Ratio Power Allocation (GRPA) and Normalized Gain Difference Power Allocation (NGDPA), achieving superior fairness, higher overall throughput, and reduced bit error rates - even under a challenging 10 dB SNR. Notably, the trained agent demonstrates strong generalization capabilities, maintaining optimal performance in unseen environments without requiring retraining. Overall, this work makes two major contributions: it presents a pioneering NOMA-CSK VLC system design and delivers a robust, adaptive power allocation solution critical for real-world applications.