Space-division multiple access (SDMA) plays an important role in modern wireless communications. Its performance depends on the channel properties, which can be improved by reconfigurable intelligent surfaces (RISs). In this work, we jointly optimize SDMA precoding at the base station (BS) and RIS configuration. We tackle difficulties of mutual coupling between RIS elements, scalability to more than 1000 RIS elements, and high requirement for channel estimation. We first derive an RIS-assisted channel model considering mutual coupling, then propose an unsupervised machine learning (ML) approach to optimize the RIS with a dedicated neural network (NN) architecture RISnet, which has good scalability, desired permutation-invariance, and a low requirement for channel estimation. Moreover, we leverage existing high-performance analytical precoding scheme to propose a hybrid solution of ML-enabled RIS configuration and analytical precoding at BS. More generally, this work is an early contribution to combine ML technique and domain knowledge in communication for NN architecture design. Compared to generic ML, the problem-specific ML can achieve higher performance, lower complexity and permutation-invariance.