This paper presents the design, modeling, and experimental validation of CapsuleBot, a novel hybrid aerial-ground bi-copter robot designed for long-endurance and low-noise operations. CapsuleBot combines the maneuverability of a bi-copter in the air with the low power consumption and low noise of a two-wheel self-balancing robot on the ground. To achieve this, we design an innovative mechanical structure named the actuated-wheel-rotor, which uses a servo motor and a brushless motor to function as both a tilting rotor in the air and an actuated wheel on the ground. CapsuleBot is equipped with two actuated-wheel-rotors, enabling it to achieve hybrid aerial-ground propulsion using only four motors, with no additional motors required compared to a bi-copter. Additionally, we develop comprehensive dynamics and control systems for both air and wheel mode, based on the bi-copter model and the two-wheel self-balancing robot model. A prototype of CapsuleBot is constructed, and its performance in terms of low power consumption and low noise is validated through experiments. Challenging tasks demonstrate CapsuleBot's capability to climb steep, fly over cliffs, and traverse rough terrains.