This paper explores a new secure wireless communication paradigm where the physical layer security technology is applied to counteract both the detection and eavesdropping attacks, such that the critical covertness and secrecy properties of the communication are jointly guaranteed. We first provide theoretical modeling for covertness outage probability (COP), secrecy outage probability (SOP) and transmission probability (TP) to depict the covertness, secrecy and transmission performances of the paradigm. To understand the fundamental security performance under the new paradigm, we then define a new metric - covert secrecy rate (CSR), which characterizes the maximum transmission rate subject to the constraints of COP, SOP and TP. We further conduct detailed theoretical analysis to identify the CSR under various scenarios determined by the detector-eavesdropper relationships and the secure transmission schemes adopted by transmitters. Finally, numerical results are provided to illustrate the achievable performances under the new secure communication paradigm.