Chemical Shift Imaging (CSI) or Chemical Shift Encoded Magnetic Resonance Imaging (CSE-MRI) enables the quantification of different chemical species in the human body, and it is one of the most widely used imaging modalities used to quantify fat in the human body. Although there have been substantial improvements in the design of signal acquisition protocols and the development of a variety of methods for the recovery of parameters of interest from the measured signal, it is still challenging to obtain a consistent and reliable quantification over the entire field of view. In fact, there are still discrepancies in the quantities recovered by different methods, and each exhibits a different degree of sensitivity to acquisition parameters such as the choice of echo times. Some of these challenges have their origin in the signal model itself. In particular, it is non-linear, and there may be different sets of parameters of interest compatible with the measured signal. For this reason, a thorough analysis of this model may help mitigate some of the remaining challenges, and yield insight into novel acquisition protocols. In this work, we perform an analysis of the signal model underlying CSI, focusing on finding suitable conditions under which recovery of the parameters of interest is possible. We determine the sources of non-identifiability of the parameters, and we propose a reconstruction method based on smooth non-convex optimization under convex constraints that achieves exact local recovery under suitable conditions. A surprising result is that the concentrations of the chemical species in the sample may be identifiable even when other parameters are not. We present numerical results illustrating how our theoretical results may help develop novel acquisition techniques, and showing how our proposed recovery method yields results comparable to the state-of-the-art.