For a well-studied family of domination-type problems, in bounded-treewidth graphs, we investigate whether it is possible to find faster algorithms. For sets $σ,ρ$ of non-negative integers, a $(σ,ρ)$-set of a graph $G$ is a set $S$ of vertices such that $|N(u)\cap S|\in σ$ for every $u\in S$, and $|N(v)\cap S|\in ρ$ for every $v\not\in S$. The problem of finding a $(σ,ρ)$-set (of a certain size) unifies common problems like $\text{Independent Set}$, $\text{Dominating Set}$, $\text{Independent Dominating Set}$, and many others. In an accompanying paper, it is proven that, for all pairs of finite or cofinite sets $(σ,ρ)$, there is an algorithm that counts $(σ,ρ)$-sets in time $(c_{σ,ρ})^{\text{tw}}\cdot n^{O(1)}$ (if a tree decomposition of width $\text{tw}$ is given in the input). Here, $c_{σ,ρ}$ is a constant with an intricate dependency on $σ$ and $ρ$. Despite this intricacy, we show that the algorithms in the accompanying paper are most likely optimal, i.e., for any pair $(σ, ρ)$ of finite or cofinite sets where the problem is non-trivial, and any $\varepsilon>0$, a $(c_{σ,ρ}-\varepsilon)^{\text{tw}}\cdot n^{O(1)}$-algorithm counting the number of $(σ,ρ)$-sets would violate the Counting Strong Exponential-Time Hypothesis ($\#$SETH). For finite sets $σ$ and $ρ$, our lower bounds also extend to the decision version, showing that those algorithms are optimal in this setting as well.