The objective of this paper is to investigate the connection between penalty functions from stochastic optimal control, convex semigroups from analysis and convex expectations from probability theory. Our main result provides a one-to-one relation between these objects. As an application, we use the representation via penality functions and duality arguments to show that convex expectations are determined by their finite dimensional distributions. To illustrate this structural result, we show that Hu and Peng's axiomatic description of $G$-Lévy processes in terms of finite dimensional distributions extends uniquely to the control approach introduced by Neufeld and Nutz. Finally, we show that convex expectations with a Markovian structure are fully determined by their one-dimensional distributions, which give rise to a classical semigroup on the state space. As an application of this result, we establish a Laplace principle for entropic risk measures associated to controlled diffusions.