This paper improves safe motion primitives-based teleoperation of a multirotor by developing a hierarchical collision avoidance method that modulates maximum speed based on environment complexity and perceptual constraints. Safe speed modulation is challenging in environments that exhibit varying clutter. Existing methods fix maximum speed and map resolution, which prevents vehicles from accessing tight spaces and places the cognitive load for changing speed on the operator. We address these gaps by proposing a high-rate (10 Hz) teleoperation approach that modulates the maximum vehicle speed through hierarchical collision checking. The hierarchical collision checker simultaneously adapts the local map's voxel size and maximum vehicle speed to ensure motion planning safety. The proposed methodology is evaluated in simulation and real-world experiments and compared to a non-adaptive motion primitives-based teleoperation approach. The results demonstrate the advantages of the proposed teleoperation approach both in time taken and the ability to complete the task without requiring the user to specify a maximum vehicle speed.