Bayesian optimization (BO) is a powerful black-box optimization framework that looks to efficiently learn the global optimum of an unknown system by systematically trading-off between exploration and exploitation. However, the use of BO as a tool for coordinated decision-making in multi-agent systems with unknown structure has not been widely studied. This paper investigates a black-box optimization problem over a multi-agent network coupled via shared variables or constraints, where each subproblem is formulated as a BO that uses only its local data. The proposed multi-agent BO (MABO) framework adds a penalty term to traditional BO acquisition functions to account for coupling between the subsystems without data sharing. We derive a suitable form for this penalty term using alternating directions method of multipliers (ADMM), which enables the local decision-making problems to be solved in parallel (and potentially asynchronously). The effectiveness of the proposed MABO method is demonstrated on an intelligent transport system for fuel efficient vehicle platooning.