This paper deals with the union set of a stationary Poisson process of cylinders in $\mathbb{R}^n$ having an $(n-m)$-dimensional base and an $m$-dimensional direction space, where $m\in\{0,1,\ldots,n-1\}$ and $n\geq 2$. The concept simultaneously generalises those of a Boolean model and a Poisson hyperplane or $m$-flat process. Under very general conditions on the typical cylinder base a Berry-Esseen bound for the volume of the union set within a sequence of growing test sets is derived. Assuming convexity of the cylinder bases and of the window a similar result is shown for a broad class of geometric functionals, including the intrinsic volumes. In this context the asymptotic variance constant is analysed in detail, which in contrast to the Boolean model leads to a new degeneracy phenomenon. A quantitative central limit theory is developed in a multivariate set-up as well.