In room acoustic environments, the Relative Transfer Functions (RTFs) are controlled by few underlying modes of variability. Accordingly, they are confined to a low-dimensional manifold. In this letter, we investigate a RTF inverse regression problem, the task of which is to generate the high-dimensional responses from their low-dimensional representations. The problem is addressed from a pure data-driven perspective and a supervised Deep Neural Network (DNN) model is applied to learn a mapping from the source-receiver poses (positions and orientations) to the frequency domain RTF vectors. The experiments show promising results: the model achieves lower prediction error of the RTF than the free field assumption. However, it fails to compete with the linear interpolation technique in small sampling distances.