The concept of $k$-defective clique, a relaxation of clique by allowing up-to $k$ missing edges, has been receiving increasing interests recently. Although the problem of finding the maximum $k$-defective clique is NP-hard, several practical algorithms have been recently proposed in the literature, with kDC being the state of the art. kDC not only runs the fastest in practice, but also achieves the best time complexity. Specifically, it runs in $O^*(γ_k^n)$ time when ignoring polynomial factors; here, $γ_k$ is a constant that is smaller than two and only depends on $k$, and $n$ is the number of vertices in the input graph $G$. In this paper, we propose the kDC-Two algorithm to improve the time complexity as well as practical performance. kDC-Two runs in $O^*( (αΔ)^{k+2} γ_{k-1}^α)$ time when the maximum $k$-defective clique size $ω_k(G)$ is at least $k+2$, and in $O^*(γ_{k-1}^n)$ time otherwise, where $α$ and $Δ$ are the degeneracy and maximum degree of $G$, respectively. In addition, with slight modification, kDC-Two also runs in $O^*( (αΔ)^{k+2} (k+1)^{α+k+1-ω_k(G)})$ time by using the degeneracy gap $α+k+1-ω_k(G)$ parameterization; this is better than $O^*( (αΔ)^{k+2}γ_{k-1}^α)$ when $ω_k(G)$ is close to the degeneracy-based upper bound $α+k+1$. Finally, to further improve the practical performance, we propose a new degree-sequence-based reduction rule that can be efficiently applied, and theoretically demonstrate its effectiveness compared with those proposed in the literature. Extensive empirical studies on three benchmark graph collections show that our algorithm outperforms the existing fastest algorithm by several orders of magnitude.