Most commercially available fixed-wing aerial vehicles (FWV) can carry only small, lightweight computing hardware such as Jetson TX2 onboard. Solving non-linear trajectory optimization on these computing resources is computationally challenging even while considering only the kinematic motion model. Most importantly, the computation time increases sharply as the environment becomes more cluttered. In this paper, we take a step towards overcoming this bottleneck and propose a trajectory optimizer that achieves online performance on both conventional laptops/desktops and Jetson TX2 in a typical urban environment setting. Our optimizer builds on the novel insight that the seemingly non-linear trajectory optimization problem for FWV has an implicit multi-convex structure. Our optimizer exploits these computational structures by bringing together diverse concepts from Alternating Minimization, Bregman iteration, and Alternating Direction Method of Multipliers. We show that our optimizer outperforms the state-of-the-art implementation of sequential quadratic programming approach in optimal control solver ACADO in computation time and solution quality measured in terms of control and goal reaching cost.