Smart Buildings are networks of connected devices and software in charge of automatically managing and controlling several building functions such as HVAC, fire alarms, lighting, shading and more. These systems evolved from mostly electronic and mechanical elements to complex systems relying on IT and wireless technologies and networks. This exposes smart buildings to new risks and threats that need to be enumerated and addressed. Research efforts have been done in several areas related to security in smart buildings but a clear overview of the research field is missing. In this paper, we present the results of a systematic literature review that provides a thorough understanding of the state of the art in research on the security of smart buildings. We found that the field of smart buildings security is growing significantly in complexity due to the many protocols introduced recently and that the research community is already studying. We also found an important lack of empirical evaluations, though evaluations on testbeds and real systems seems to be growing. Finally, we found an almost complete lack of consideration of non-technical aspects, such as social, organisational, and human factors, which are crucial in this type of systems, where ownership and liability is not always clear.