Distribution system state estimation (DSSE) is a core task for monitoring and control of distribution networks. Widely used algorithms such as Gauss-Netwon perform poorly with the limited number of measurements typically available for DSSE, often require many iterations to obtain reasonable results, and sometimes fail to converge. DSSE is a non-convex problem, and working with a limited number of measurements further aggravate the situation, as indeterminacy induces multiple global (in addition to local) minima. Gauss-Newton is also known to be sensitive to initialization. Hence, the situation is far from ideal. It is therefore natural to ask if there is a smart way of initializing Gauss-Newton that will avoid these DSSE-specific pitfalls. This paper proposes using historical or simulation-derived data to train a shallow neural network to `learn to initialize' -- that is, map the available measurements to a point in the neighborhood of the true latent states (network voltages), which is used to initialize Gauss-Newton. It is shown that this hybrid machine learning / optimization approach yields superior performance in terms of stability, accuracy, and runtime efficiency, compared to conventional optimization-only approaches. It is also shown that judicious design of the neural network training cost function helps to improve the overall DSSE performance.