In the last twenty years network science has proven its strength in modelling many real-world interacting systems as generic agents, the nodes, connected by pairwise edges. Yet, in many relevant cases, interactions are not pairwise but involve larger sets of nodes, at a time. These systems are thus better described in the framework of hypergraphs, whose hyperedges effectively account for multi-body interactions. We hereby propose a new class of random walks defined on such higher-order structures, and grounded on a microscopic physical model where multi-body proximity is associated to highly probable exchanges among agents belonging to the same hyperedge. We provide an analytical characterisation of the process, deriving a general solution for the stationary distribution of the walkers. The dynamics is ultimately driven by a generalised random walk Laplace operator that reduces to the standard random walk Laplacian when all the hyperedges have size 2 and are thus meant to describe pairwise couplings. We illustrate our results on synthetic models for which we have a full control of the high-order structures, and real-world networks where higher-order interactions are at play. As a first application of the method, we compare the behaviour of random walkers on hypergraphs to that of traditional random walkers on the corresponding projected networks, drawing interesting conclusions on node rankings in collaboration networks. As a second application, we show how information derived from the random walk on hypergraphs can be successfully used for classification tasks involving objects with several features, each one represented by a hyperedge. Taken together, our work contributes to unveiling the effect of higher-order interactions on diffusive processes in higher-order networks, shading light on mechanisms at the hearth of biased information spreading in complex networked systems.