In an earlier paper, a randomized load balancing model was studied in a heavy traffic asymptotic regime where the load balancing stream is thin compared to the total arrival stream. It was shown that the limit is given by a system of rank-based Brownian particles on the half-line. This paper extends these results from the case of exponential service time to an invariance principle, where service times have finite second moment. The main tool is a new notion of rank-based stochastic differential inclusion, which may be of interest in its own right.