We show that various functionals related to the supremum of a real function defined on an arbitrary set or a measure space are Hadamard directionally differentiable. We specifically consider the supremum norm, the supremum, the infimum, and the amplitude of a function. The (usually non-linear) derivatives of these maps adopt simple expressions under suitable assumptions on the underlying space. As an application, we improve and extend to the multidimensional case the results in \cite{Raghavachari} regarding the limiting distributions of Kolmogorov-Smirnov type statistics under the alternative hypothesis. Similar results are obtained for analogous statistics associated with copulas. We additionally solve an open problem about the Berk-Jones statistic proposed by \cite{Jager-Wellner-2004}. Finally, the asymptotic distribution of maximum mean discrepancies over Donsker classes of functions is derived.