Social engineering attacks are phenomena that are equally applicable to both the physical world and cyberspace. These attacks in the physical world have been studied for a much longer time than their counterpart in cyberspace. This motivates us to investigate how social engineering attacks in the physical world and cyberspace relate to each other, including their common characteristics and unique features. For this purpose, we propose a methodology to unify social engineering attacks and defenses in the physical world and cyberspace into a single framework, including: (i) a systematic model based on psychological principles for describing these attacks; (ii) a systematization of these attacks; and (iii) a systematization of defenses against them. Our study leads to several insights, which shed light on future research directions towards adequately defending against social engineering attacks in cyberspace.