Many high-dimensional uncertainty quantification problems are solved by polynomial dimensional decomposition (PDD), which represents Fourier-like series expansion in terms of random orthonormal polynomials with increasing dimensions. This study constructs dimension-wise and orthogonal splitting of polynomial spaces, proves completeness of polynomial orthogonal basis for prescribed assumptions, and demonstrates mean-square convergence to the correct limit -- all associated with PDD. A second-moment error analysis reveals that PDD cannot commit larger error than polynomial chaos expansion (PCE) for the appropriately chosen truncation parameters. From the comparison of computational efforts, required to estimate with the same precision the variance of an output function involving exponentially attenuating expansion coefficients, the PDD approximation can be markedly more efficient than the PCE approximation.