[Context] Although software development is an inherently human activity, research in software engineering (SE) has long focused mostly on processes and tools, failing to recall about the human factors behind. Even when explored, researchers typically do not properly use psychology background to better understand human factors in SE, such as the psychometric instruments, which aim to measure human factors. [Objective] Our goal is to provide a critical review on the use of psychometric instruments in SE research regarding personality. [Method] We present a two-step study. First, a systematic mapping of the literature in order to generate a catalog of the psychometric instruments used; second, a preliminary survey to be conducted with social sciences researchers to assess their adoption in SE research. [Results and Conclusion] The results so far are quite initial. The next steps direct us to finish the data extraction to finalize the catalog (systematic mapping) and to refine the survey design and apply it with social sciences researchers.