We study the contraction properties of a quasi-posterior distribution $\checkΠ_{n,d}$ obtained by combining a quasi-likelihood function and a sparsity inducing prior distribution on $\rset^d$, as both $n$ (the sample size), and $d$ (the dimension of the parameter) increase. We derive some general results that highlight a set of sufficient conditions under which $\checkΠ_{n,d}$ puts increasingly high probability on sparse subsets of $\rset^d$, and contracts towards the true value of the parameter. We apply these results to the analysis of logistic regression models, and binary graphical models, in high-dimensional settings. For the logistic regression model, we shows that for well-behaved design matrices, the posterior distribution contracts at the rate $O(\sqrt{s_\star\log(d)/n})$, where $s_\star$ is the number of non-zero components of the parameter. For the binary graphical model, under some regularity conditions, we show that a quasi-posterior analog of the neighborhood selection of \cite{meinshausen06} contracts in the Frobenius norm at the rate $O(\sqrt{(p+S)\log(p)/n})$, where $p$ is the number of nodes, and $S$ the number of edges of the true graph.