We consider the problem of inference for local parameters of a convex regression function $f_0: [0,1] \to \mathbb{R}$ based on observations from a standard nonparametric regression model, using the convex least squares estimator (LSE) $\widehat{f}_n$. For $x_0 \in (0,1)$, the local parameters include the pointwise function value $f_0(x_0)$, the pointwise derivative $f_0'(x_0)$, and the anti-mode (i.e., the smallest minimizer) of $f_0$. The existing limiting distribution of the estimation error $(\widehat{f}_n(x_0) - f_0(x_0), \widehat{f}_n'(x_0) - f_0'(x_0) )$ depends on the unknown second derivative $f_0''(x_0)$, and is therefore not directly applicable for inference. To circumvent this impasse, we show that the following locally normalized errors (LNEs) enjoy pivotal limiting behavior: Let $[\widehat{u}(x_0), \widehat{v}(x_0)]$ be the maximal interval containing $x_0$ where $\widehat{f}_n$ is linear. Then, under standard conditions, $$\binom{ \sqrt{n(\widehat{v}(x_0)-\widehat{u}(x_0))}(\widehat{f}_n(x_0)-f_0(x_0)) }{ \sqrt{n(\widehat{v}(x_0)-\widehat{u}(x_0))^3}(\widehat{f}_n'(x_0)-f_0'(x_0))} \rightsquigarrow σ\cdot \binom{\mathbb{L}^{(0)}_2}{\mathbb{L}^{(1)}_2},$$ where $n$ is the sample size, $σ$ is the standard deviation of the errors, and $\mathbb{L}^{(0)}_2, \mathbb{L}^{(1)}_2$ are universal random variables. This asymptotically pivotal LNE theory instantly yields a simple tuning-free procedure for constructing CIs with asymptotically exact coverage and optimal length for $f_0(x_0)$ and $f_0'(x_0)$. We also construct an asymptotically pivotal LNE for the anti-mode of $f_0$, and its limiting distribution does not even depend on $σ$. These asymptotically pivotal LNE theories are further extended to other convexity/concavity constrained models (e.g., log-concave density estimation) for which a limit distribution theory is available for problem-specific estimators.