We study the Poisson Boolean model with convex bodies which are rotation-invariant distributed. We assume that the convex bodies have regularly varying diameters with indices $-α_1\geq \dots\geq-α_d$ where $α_k >0$ for all $k\in\{1,\dots,d\}.$ It is known that a sufficient condition for the robustness of the model, i.e. the union of the convex bodies has an unbounded connected component no matter what the intensity of the underlying Poisson process is, is that there exists some $k\in\{1,\dots,d\}$ such that $α_k<\min\{2k,d\}$. To avoid that this connected component covers all of $\mathbb{R}^d$ almost surely we also require $α_k> k$ for all $k\in\{1,\dots,d\}$. We show that under these assumptions, the chemical distance of two far apart vertices $\mathbf{x}$ and $\mathbf{y}$ behaves like $c\log\log|x-y|$ as $|x-y|\rightarrow \infty$, with an explicit and very surprising constant $c$ that depends only on the model parameters. We furthermore show that if there exists $k$ such that $α_k\leq k$, the chemical distance is smaller than $c\log\log|x-y|$ for all $c>0$ and that if $α_k\geq\min\{2k,d\}$ for all $k$, it is bigger than $c\log\log|x-y|$ for all $c>0$.