In software development, privacy preservation has become essential with the rise of privacy concerns and regulations such as GDPR and CCPA. While several tools, guidelines, methods, methodologies, and frameworks have been proposed to support developers embedding privacy into software applications, most of them are proofs-of-concept without empirical evaluations, making their practical applicability uncertain. These solutions should be evaluated for different types of scenarios (e.g., industry settings such as rapid software development environments, teams with different privacy knowledge, etc.) to determine what their limitations are in various industry settings and what changes are required to refine current solutions before putting them into industry and developing new developer-supporting approaches. For that, a thorough review of empirically evaluated current solutions will be very effective. However, the existing secondary studies that examine the available developer support provide broad overviews but do not specifically analyze empirically evaluated solutions and their limitations. Therefore, this Systematic Literature Review (SLR) aims to identify and analyze empirically validated solutions that are designed to help developers in privacy-preserving software development. The findings will provide valuable insights for researchers to improve current privacy-preserving solutions and for practitioners looking for effective and validated solutions to embed privacy into software development.