We consider controlled stochastic differential equations (SDEs) with measurable coefficients, a uniformly elliptic diffusion coefficient and an $L_d$-drift. No space-regularity will be assumed for the coefficients. In this framework we investigate the relation of value functions, partial differential equations (PDEs) and operator semigroups. First, for a cost with infinite time horizon on a bounded domain, we identify the value function as $L_{d_0}$-viscosity solution to a Hamilton-Jacobi-Bellman equation and we establish quantitative regularity estimates. The constant $d_0 \in (d/2, d)$ only depends on the space dimension $d$, the ellipticity constants of the diffusion coefficient and the $L_d$-bound of the drift. To illustrate applications of these results, we provide a uniqueness theorem under an additional assumption on the diffusion coefficient, showing a stochastic representation, and we discuss stability of value functions. Second, we consider a cost with a finite time horizon, terminal and running terms. We show that the value function indexed over the terminal cost is a nonlinear semigroup on $C_b (\mathbb{R}^d)$ and we establish a regularization by noise effect, which shows that the semigroup regularizes lower semicontinuity to local Hölder continuity. Lastly, we relate the semigroup to a parabolic PDE, showing that it is an $L_{d + 1}$-viscosity solution, and we establish local in time and global in space quantitative regularity estimates. Our proofs for the regularity of the value functions, the $C_b$-Feller property of the semigroup and its regularization by noise effects are based on a strong Markov selection principle and analytic estimates for linear diffusions that were recently established by N. V. Krylov in a series of papers. We highlight that our method covers frameworks without uniqueness of the controlled SDEs, as well as the associated PDEs.