Join order optimization is among the most crucial query optimization problems, and its central position is also evident in the new research field where quantum computing is applied to database optimization and data management. In the field, join order optimization is the most studied database problem, usually tackled with a quadratic unconstrained binary optimization model, which is solved with various meta-heuristics such as quantum annealing, quantum approximate optimization algorithm, or variational quantum eigensolver. In this work, we continue developing quantum computing techniques for join order optimization by presenting three novel quantum optimization algorithms. These algorithms are based on a higher-order unconstrained binary optimization model, which is a generalization of the quadratic model and has not previously been applied to database problems. Theoretically, these optimization problems naturally map to universal quantum computers and quantum annealers. Compared to previous research, two of our algorithms are the first quantum algorithms to precisely model the join order cost function. We prove theoretical bounds by showing that these two methods encode the same plans as the dynamic programming algorithm without cross-products, which provides the optimal result up to cross-products. The third algorithm reaches at least as good plans as the greedy algorithm without cross-products. These results set an important theoretical connection between the classical and quantum algorithms for join order selection, which has not been studied in the previous research. To demonstrate our algorithms' practical usability, we have conducted an experimental evaluation on thousands of clique, cycle, star, tree, and chain query graphs using quantum and classical solvers.