Let $X_1,\ldots,X_n$ be a random sample from the Gamma distribution with density $f(x)=λ^αx^{α-1}e^{-λx}/Γ(α)$, $x>0$, where both $α>0$ (the shape parameter) and $λ>0$ (the reciprocal scale parameter) are unknown. The main result shows that the uniformly minimum variance unbiased estimator (UMVUE) of the shape parameter, $α$, exists if and only if $n\geq 4$; moreover, it has finite variance if and only if $n\geq 6$. More precisely, the form of the UMVUE is given for all parametric functions $α$, $λ$, $1/α$ and $1/λ$. Furthermore, a highly efficient estimating procedure for the two-parameter Beta distribution is also given. This is based on a Stein-type covariance identity for the Beta distribution, followed by an application of the theory of $U$-statistics and the delta-method. MSC: Primary 62F10; 62F12; Secondary 62E15. Key words and phrases: unbiased estimation; Gamma distribution; Beta distribution; Ye-Chen-type closed-form estimators; asymptotic efficiency; $U$-statistics; Stein-type covariance identity; delta-method.