We look at stochastic optimization problems through the lens of statistical decision theory. In particular, we address admissibility, in the statistical decision theory sense, of the natural sample average estimator for a stochastic optimization problem (which is also known as the empirical risk minimization (ERM) rule in learning literature). It is well known that for some simple stochastic optimization problems, the sample average estimator may not be admissible. This is known as {\em Stein's paradox} in the statistics literature. We show in this paper that for optimizing stochastic linear functions over compact sets, the sample average estimator is admissible. Moreover, we study problems with convex quadratic objectives subject to box constraints. Stein's paradox holds when there are no constraints and the dimension of the problem is at least three. We show that in the presence of box constraints, admissibility is recovered for dimensions 3 and 4.