The contact process with an asymptomatic state, introduced in [Belhadji, Lanchier and Mercer, Stochastic Process. Appl., 176:104417, 2024], is a natural variant of the basic contact process that distinguishes between asymptomatic (state 1) and symptomatic (state 2) individuals. Infected individuals infect their healthy neighbors at rate $λ_1$ when asymptomatic and at rate $λ_2$ when symptomatic. Newly infected individuals are always asymptomatic and become symptomatic at rate $γ$, and infected individuals recover at rate one regardless of whether they are asymptomatic or symptomatic. Belhadji, Lanchier and Mercer proved that, in the mean-field approximation, there is an epidemic if and only if $λ_1 + γλ_2 > 1 + γ$, showing in particular that, for all $γ> 0$, there is an epidemic for $λ_2$ sufficiently large. In contrast, comparing the process with a subcritical Galton-Watson branching process, they proved for the spatial model that, if $γ< 1 / (4d - 1)$ and $λ_1 = 0$, then there is no epidemic even in the limiting case $λ_2 = \infty$. In this paper, we prove an exponential decay of the progeny of the Galton-Watson branching process, and use a block construction and a perturbation argument, to extend the extinction phase of the process to $λ_1 > 0$ small.