This paper focuses on developing a bearings-only measurement-based three-dimensional window traversal guidance method for quadrotor Uninhabitated Aerial Vehicles (UAVs). The desired flight path and heading angles of the quadrotor are proposed as functions of the bearing angle information of the four vertices of the window. These angular guidance inputs employ a bearing angle bisector term and an elliptic shaping angle term, which directs the quadrotor towards the centroid of the window. Detailed stability analysis of the resulting kinematics demonstrates that all quadrotor trajectories lead to the centroid of the window along a direction which is normal to the window plane. A qualitative comparison with existing traversal methodologies showcases the superiority of the proposed guidance approach with regard to the nature of information, computations for generating the guidance commands, and flexibility of replanning the traversal path. Realistic simulations considering six degree-of-freedom quadrotor model and Monte Carlo studies validate the effectiveness, accuracy, and robustness of the proposed guidance solution. Representative flight validation trials are carried out using an indoor motion capture system.