Temporal higher-order networks, where each hyperlink involving a group of nodes are activated or deactivated over time, are recently used to represent complex systems such as social contacts, interactions or collaborations that occur at specific times. Such networks are substrates for social contagion processes like the diffusion of information and opinions. In this work, we consider eight temporal higher-order networks derived from human face-to-face interactions in various contexts and the Susceptible-Infected threshold process on each of these networks: whenever a hyperlink is active and the number of infected nodes in the hyperlink exceeds a threshold $Θ$, each susceptible node in the hyperlink is infected independently with probability $β$. The objective is to understand (1) the contribution of each hyperlink to the diffusion process, namely, the average number of nodes that are infected directly via the activation of the hyperlink when the diffusion starts from an arbitrary seed node, and (2) hyperlinks with what network properties tend to contribute more. We first propose to construct the diffusion backbone. The backbone is a weighted higher-order network, where the weight of each hyperlink denotes the contribution of the hyperlink to a given diffusion process. Secondly, we find that the backbone, or the contribution of hyperlinks, is dependent on the parameters $β$ and $Θ$ of the diffusion process, which is also supported by our theoretical analysis of the backbone when $β\rightarrow 0$. Thirdly, we systematically design centrality metrics for hyperlinks in a temporal higher-order network, and each centrality metric is used to estimate the ranking of hyperlinks by the weight in the backbone. Finally, we find and explain why different centrality metrics can better estimate the contributions of hyperlinks for different parameters of the diffusion process.