An infinitely divisible distribution on $\mathbb{R}$ is a probability measure $μ$ such that the characteristic function $\hatμ$ has a Lévy-Khintchine representation with characteristic triplet $(a,γ, ν)$, where $ν$ is a Lévy measure, $γ\in\mathbb{R}$ and $a\ge 0$. A natural extension of such distributions are quasi-infinitely distributions. Instead of a Lévy measure, we assume that $ν$ is a "signed Lévy measure", for further information on the definition see [\ref{Lindner}]. We show that a distribution $μ=pδ_{x_0}+(1-p)μ_{ac}$ with $p>0$ and $x_0 \in \mathbb{R}$, where $μ_{ac}$ is the absolutely continuous part, is quasi-infinitely divisible if and only if $\hatμ(z)\neq0$ for every $z\in\mathbb{R}$. We apply this to show that certain variance mixtures of mean zero normal distributions are quasi-infinitely divisible distributions, and we give an example of a quasi-infinitely divisible distribution that is not continuous but has infinite quasi-Lévy measure. Furthermore, it is shown that replacing the signed Lévy measure by a seemingly more general complex Lévy measure does not lead to new distributions. Last but not least it is proven that the class of quasi-infinitely divisible distributions is not open, but path-connected in the space of probability measures with the Prokhorov metric.