Aerial autonomy stack (AAS) is a batteries included software stack to:
- Develop multi-drone autonomy—with ROS2, PX4, and ArduPilot
- Simulate faster-than-real-time perception and control—with YOLO and 3D LiDAR
- Deploy in real drones—with JetPack, DeepStream, and NVIDIA Orin
For an example bill of materials, read BOM.md; for motivation, read RATIONALE.md
aerial-autonomy-stack-v3.mp4
Features (click to expand)
- PX4 and ArduPilot multi-vehicle simulation (quadrotors and VTOLs)
- ROS2 action-based autopilot interface (via XRCE-DDS or MAVROS)
- YOLO (with ONNX GPU Runtimes) and LiDAR Odometry (with KISS-ICP)
- 3D worlds for perception-based simulation
- Steppable Gymnasium environment and faster-than-real-time, multi-instance simulation
- Gazebo's wind effects and waves plugins
- Dockerized simulation based on Ubuntu with CUDA and cuDNN
- Dockerized deployment based on NVIDIA JetPack with DeepStream
- Windows 11 compatibility via WSL
- Multi-Jetson-in-the-loop (HITL) simulation to test NVIDIA- and ARM-based on-board compute
- Dual network to separate simulated sensors (
SIM_SUBNET) and inter-vehicle comms (AIR_SUBNET) - Zenoh inter-vehicle ROS2 bridge
- PX4 Offboard interface (e.g. CTBR/
VehicleRatesSetpointfor agile, GNSS-denied flight) - ArduPilot Guided interface (i.e.
setpoint_velocity,setpoint_accelreferences) - Logs analysis with
flight_review(.ulg), MAVExplorer (.bin), and PlotJuggler (rosbag)
1. Installation
AAS is developed on Ubuntu 24.04 with
nvidia-driver-580using an i7-11 with 16GB RAM and RTX 3060Read
REQUIREMENTS_UBUNTU.md(orREQUIREMENTS_WSL.mdfor Windows 11) to install the requirements
sudo apt update && sudo apt install -y git xterm xfonts-base wget unzip git clone https://github.com/JacopoPan/aerial-autonomy-stack.git && cd aerial-autonomy-stack/tools_and_docs/ ./tests/check_requirements.sh # AAS requires nvidia-driver-580, docker, and nvidia-container-toolkit ./sim_build.sh # The 1st build takes ~45' with good internet (`Ctrl + c` and restart if needed, cached stages will be preserved)
ghcr.io pre-built images (click to expand)
# ghcr.io images are re-built from `main` every Friday night for name in aircraft ground simulation; do docker pull ghcr.io/jacopopan/${name}-image:latest docker tag ghcr.io/jacopopan/${name}-image:latest ${name}-image:latest done
2. Multi-drone Simulation
Start AAS:
cd aerial-autonomy-stack/tools_and_docs/ AUTOPILOT=px4 NUM_QUADS=1 NUM_VTOLS=1 WORLD=swiss_town HEADLESS=false RTF=3.0 ./sim_run.sh # Start a simulation, check the script for more options (note: ArduPilot SITL checks take ~30s of simulated time before being ready to arm)
There are 3 main ways to autonomously fly the drones (plus QGroundControl for operator supervision)
- From the
Ground's Xterm terminal, fly all drones in a synchronized formation withdtc_controller_node:
ros2 run drone_traffic_controller dtc_controller --ros-args -p use_sim_time:=true
- From any
QUAD/VTOLXterm terminal, fly a behavior tree mission (e.g.,yalla.yaml):
ros2 run mission mission --conops yalla.yaml --ros-args -r __ns:=/Drone$DRONE_ID -p use_sim_time:=true- From any
QUAD/VTOLXterm terminal, use ROS2 actions forpx4_offboard/ardupilot_guided:
cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/takeoff_action \ autopilot_interface_msgs/action/Takeoff '{takeoff_altitude: 30.0}'" # Press Enter to cancel the action or regain the terminal when it finishes cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/offboard_action \ autopilot_interface_msgs/action/Offboard \ '{controller_name: att-test, max_duration_sec: 10.0}'" # Add or re-implement offboard controllers in `px4_offboard.cpp`, `ardupilot_guided.cpp`
./sim_run.sh options:
- AUTOPILOT=px4, ardupilot
- HEADLESS/CAMERA/LIDAR=true, false
- NUM_QUADS/NUM_VTOLS=0, 1, ...
- WORLD=impalpable_greyness, apple_orchard, shibuya_crossing, swiss_town, waterworld
- RTF=1.0, 2.0, ... (real-time-factor, use 0.0 for "as fast as possible)
- INSTANCE=0, 1, ... (integer ID to run multiple parallel simulations)
WORLDs: (i)apple_orchard, a GIS world created using BlenderGIS / (ii)impalpable_greyness, an empty world with simple shapes / (iii)shibuya_crossing, a 3D world adapted from cgtrader / (iv)swiss_town, a photogrammetry world courtesy of Pix4D / pix4d.com / (v)waterworld, a dynamic world using theasv_wave_simwave plugin
Tip
Edit sensor_config.yaml, then run sim_build.sh, to customize the sensor parameters
Use ROS2 drone and gimbal control primitives from CLI (click to expand)
# Takeoff action (quads and VTOLs) cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/takeoff_action autopilot_interface_msgs/action/Takeoff '{takeoff_altitude: 40.0, vtol_transition_heading: 330.0, vtol_loiter_nord: 200.0, vtol_loiter_east: 100.0, vtol_loiter_alt: 120.0}'" # Land (at home) action (quads and VTOLs) cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/land_action autopilot_interface_msgs/action/Land '{landing_altitude: 60.0, vtol_transition_heading: 60.0}'" # Orbit action (quads and VTOLs) cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/orbit_action autopilot_interface_msgs/action/Orbit '{east: 500.0, north: 0.0, altitude: 150.0, radius: 200.0}'" # Reposition service (quads only) ros2 service call /Drone${DRONE_ID}/set_reposition autopilot_interface_msgs/srv/SetReposition '{east: 50.0, north: 100.0, altitude: 60.0}' # Offboard action (Specify the flight behavior via `controller_name`, e.g., "traj-test" for PX4 or "vel-test" for ArduPilot) cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/offboard_action autopilot_interface_msgs/action/Offboard '{controller_name: traj-test, max_duration_sec: 5.0}'" # SetSpeed service (always limited by the autopilot params, for quads applies from the next command, not effective on ArduPilot VTOLs) ros2 service call /Drone${DRONE_ID}/set_speed autopilot_interface_msgs/srv/SetSpeed '{speed: 3.0}' # Gimbal status and position control (in radians) ros2 topic echo /gimbal_state ros2 topic pub -1 /gimbal_pitch_cmd std_msgs/msg/Float64 "{data: 1.57}"
To analyze the flight logs, in the Simulation's Xterm terminal:
/aas/simulation_resources/scripts/plot_logs.sh # Analyze the flight logs at http://10.42.90.100:5006/browse or in MAVExplorerAdd or disable wind effects, in the Simulation's Xterm terminal (click to expand)
python3 /aas/simulation_resources/scripts/gz_wind.py --from_west 0.0 --from_south 3.0 python3 /aas/simulation_resources/scripts/gz_wind.py --stop_wind
Develop within live containers (click to expand)
Launching the sim_run.sh script with DEV=true, does not start the simulation and mounts folders [aircraft|ground|simulation]_resources, [aircraft|ground]_ws/src as volumes to more easily track, commit, push changes while building and testing them within the containers:
cd aerial-autonomy-stack/tools_and_docs/ DEV=true ./sim_run.sh # Starts one simulation-image, one ground-image, and one aircraft-image where the *_resources/ and *_ws/src/ folders are mounted from the host
To build changes—made on the host—in the Ground or QUAD Xterm terminal:
cd /aas/aircraft_ws/ # Or cd /aas/ground_ws/ colcon build --symlink-install
To start the simulation, in the QUAD Xterm terminal:
tmuxinator start -p /aas/aircraft.yml.erb
In the Ground Xterm terminal:
tmuxinator start -p /aas/ground.yml.erb
In the Simulation Xterm terminal:
tmuxinator start -p /aas/simulation.yml.erb
To end the simulation, in each terminal detach Tmux with Ctrl + b, then d; kill all lingering processes with tmux kill-server && pkill -f gz
3. Deployment on Jetson
AAS is tested on a Holybro Jetson Baseboard with Pixhawk 6X and NVIDIA Orin NX 16GB
The default quad is a Holybro X650 with the IMX219 camera and the Livox Mid-360S LiDAR
Read
SETUP_AVIONICS.mdandBOM.mdto setup the requirements on the Jetson and configure the PixhawkRead
SETUP_CHRONY.mdto let the Jetson timesync to theground-imagecomputer when w/o internet
sudo apt update && sudo apt install -y git git clone https://github.com/JacopoPan/aerial-autonomy-stack.git && cd aerial-autonomy-stack/tools_and_docs/ ./deploy_build.sh # Build for arm64, on Jetson Orin NX the first build takes ~50', including building onnxruntime-gpu with TensorRT support from source
On a Jetson Orin, start the aircraft-image:
cd aerial-autonomy-stack/tools_and_docs/ AUTOPILOT=px4 DRONE_ID=1 CAMERA=true LIDAR=false AIR_SUBNET=10.223 HEADLESS=true ./deploy_run.sh # The 1st run of `./deploy_run.sh` requires ~10' to build the FP16 TensorRT cache
./deploy_run.sh options:
- DRONE_TYPE=quad, vtol
- AUTOPILOT=px4, ardupilot
- DRONE_ID=1, 2, ... (ROS_DOMAIN_ID of the drone, matching the MAV_SYS_ID/SYSID_THISMAV of the autpilot)
- HEADLESS/CAMERA/LIDAR=true, false
On a laptop, start the ground-image (QGC, Zenoh, SSH, and GStreamer):
cd aerial-autonomy-stack/tools_and_docs/ ./sim_build.sh # Build all images for amd64, including ground-image GROUND=true NUM_QUADS=1 AIR_SUBNET=10.223 HEADLESS=false ./deploy_run.sh
HITL Simulation (click to expand)
Note: HITL simulation validates the Jetson compute and the inter-vehicle network. Use USB2.0 ASIX Ethernet adapters to add multiple network interfaces to the Jetson baseboards
Set up a LAN on an arbitrary SIM_SUBNET with netmask 255.255.0.0 (e.g. 172.30.x.x) between:
- One simulation computer, with IP
[SIM_SUBNET].90.100 - One ground computer, with IP
[SIM_SUBNET].90.101 NJetson Baseboards with IPs[SIM_SUBNET].90.1, ...,[SIM_SUBNET].90.N
Optionally, set up a second LAN :
AIR_SUBNETwith netmask255.255.0.0(e.g.10.223.x.x) between:
- One ground computer, with IP
[AIR_SUBNET].90.101NJetson Baseboards with IPs[AIR_SUBNET].90.1, ...,[AIR_SUBNET].90.N
First, start all aircraft containers, one on each Jetson (e.g. via SSH):
# On the Jetson with IPs ending in 90.1 HITL=true DRONE_ID=1 SIM_SUBNET=172.30 AIR_SUBNET=10.223 ./deploy_run.sh # Add HEADLESS=false if a screen is connected to the Jetson
# On the Jetson with IPs ending in 90.2 HITL=true DRONE_ID=2 SIM_SUBNET=172.30 AIR_SUBNET=10.223 ./deploy_run.sh # Add HEADLESS=false if a screen is connected to the Jetson
Then, start the simulation:
# On the computer with IPs ending in 90.100
HITL=true NUM_QUADS=2 SIM_SUBNET=172.30 ./sim_run.shFinally, start QGC and the Zenoh bridge:
# On the computer with IPs ending in 90.101
HITL=true GROUND=true NUM_QUADS=2 AIR_SUBNET=10.223 HEADLESS=false ./deploy_run.shNote: running only the first 3 commands with
GND_CONTAINER=falseputs the Zenoh bridge on theSIM_SUBNET, removing the need for the optionalAIR_SUBNETand the computer with IP ending in90.101
Once done, detach Tmux (and remove the containers) with Ctrl + b, then d
4. Gymnasium RL Environment
Using a Python venv or a conda environment is optional but recommended (click to expand)
wget https://repo.anaconda.com/archive/Anaconda3-2025.12-2-Linux-x86_64.sh # Or a newer version in https://repo.anaconda.com/archive/ bash Anaconda3-2025.12-2-Linux-x86_64.sh # Install; start a new terminal conda config --set auto_activate_base false # Turn off auto initialization of (base); start a new terminal conda update --all -n base -c defaults # Update to the latest conda version conda create -n aas python=3.12 # Latest Python version beyond "bugfix" status https://devguide.python.org/versions/
Install the aas-gym package (after completing the steps in "Installation"):
conda activate aas # If using Anaconda cd aerial-autonomy-stack/aas-gym/ pip3 install -e .
Examples:
conda activate aas # If using Anaconda cd aerial-autonomy-stack/tools_and_docs python3 gym_run.py --mode step # Manually step AAS @1Hz python3 gym_run.py --mode speedup # Speed-up test @50Hz python3 gym_run.py --mode vectorenv-speedup # Vectorized speed-up test @50Hz
Appendix A: Citation
@INPROCEEDINGS{panerati2026aas, author={Jacopo Panerati and Sina Sajjadi and Sina Soleymanpour and Varunkumar Mehta and Iraj Mantegh}, booktitle={2026 International Conference on Unmanned Aircraft Systems (ICUAS)}, title={{aerial-autonomy-stack}---a faster-than-real-time, autopilot-agnostic, {ROS2} framework to simulate and deploy perception-based drones}, year={2026}}
Appendix B: Architecture
%%{init: {'theme': 'base', 'themeVariables': { 'fontFamily': 'monospace'}}}%%
flowchart TB
subgraph aas [" "]
subgraph sim ["#nbsp;simulation#nbsp;container#nbsp;(amd64)"]
sitl("[N x] PX4 || <br/> ArduPilot SITL"):::resource
gz(Gazebo Sim):::resource
subgraph models [Models]
drones(aircraft_models/):::resource
worlds(simulation_worlds/):::resource
end
drones --> gz
worlds --> gz
sitl <--> |"gz_bridge || ardupilot_gazebo"| gz
end
subgraph gnd ["#nbsp;ground#nbsp;container#nbsp;(amd64)"]
mlrouter{{mavlink-router}}:::bridge
ground_system(ground_system):::algo
dtc_controller(dtc_controller):::algo
qgc(QGroundControl):::resource
zenoh_gnd{{zenoh-bridge}}:::bridge
ground_system --> |"/tracks"| zenoh_gnd
dtc_controller --> |"/dtc_commands"| zenoh_gnd
mlrouter <--> qgc
mlrouter --> ground_system
end
subgraph air ["[N#nbsp;x]#nbsp;aircraft#nbsp;container(s)#nbsp;(amd64,#nbsp;arm64)"]
subgraph perception [Perception]
yolo_py(yolo_py):::algo
kiss_icp(kiss_icp):::algo
livo_pkgs(livo_pkgs):::algo
end
zenoh_air{{zenoh-bridge}}:::bridge
subgraph control [Control]
dtc_client(dtc_client):::algo
mission(mission):::algo
offboard_control(offboard_control):::algo
autopilot_interface(autopilot_interface):::algo
state_sharing(state_sharing):::algo
end
detection_split( ):::splitNode
track_split( ):::splitNode
ap_link{{"uxrce_dds <br/> || MAVROS"}}:::bridge
kiss_icp -.-> |"/TBD"| ap_link
livo_pkgs <-.-> |"/imu_data <br/> /TBD"| ap_link
zenoh_air --> |"/tracks <br/> /state_drone_N"| track_split
track_split --> offboard_control
track_split --> mission
zenoh_air --> |"/dtc_commands"| dtc_client
ap_link --> state_sharing
ap_link <--> autopilot_interface
yolo_py --> |"/detections"| detection_split
detection_split --> offboard_control
detection_split --> mission
offboard_control --> |"/ctrl_ref"| autopilot_interface
mission --> |"ros2 action"| autopilot_interface
dtc_client --> |"ros2 action"| autopilot_interface
zenoh_air <--> |"/state_drone_n"| state_sharing
autopilot_interface ~~~ state_sharing
end
lidar_split( ):::splitNode
camera_split( ):::splitNode
end
gz --> |"/lidar_points <br/> [SIM_SUBNET]"| lidar_split
lidar_split --> kiss_icp
lidar_split --> livo_pkgs
gz --> |"gz_gst_bridge <br/> [SIM_SUBNET]"| camera_split
camera_split --> livo_pkgs
camera_split --> yolo_py
sitl <--> |"UDP <br/> [SIM_SUBNET]"| ap_link
sitl <--> |"MAVLink <br/> [SIM_SUBNET]"| mlrouter
zenoh_gnd <-.-> |"TCP <br/> [AIR_SUBNET]"| zenoh_air
classDef bridge fill:#ffebd6,stroke:#f5a623,stroke-width:2px;
classDef algo fill:#e1f5fe,stroke:#0277bd,stroke-width:2px;
classDef resource fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px;
classDef splitNode fill:#cccccc,stroke:#666666,stroke-width:2px;
classDef blueStyle fill:#e1f0ff,stroke:#666,stroke-width:2px; class aas blueStyle;
classDef whiteStyle fill:#f9f9f9,stroke:#666,stroke-width:1px,stroke-dasharray: 5 5; class air,gnd,sim whiteStyle;
classDef greyStyle fill:#eeeeee,stroke:#666,stroke-width:1px,stroke-dasharray: 5 5; class perception,control,models greyStyle;
linkStyle 23,24,25,26,27,28,29,30 stroke:teal,stroke-width:3px; linkStyle 31 stroke:blue,stroke-width:4px;
Repository structure (click to expand)
aerial-autonomy-stack │ ├── aas-gym │ └── src │ └── aas_gym │ └── aas_env.py # aerial-autonomy-stack as a Gymnasium environment │ ├── aircraft │ ├── aircraft_ws │ │ └── src │ │ ├── autopilot_interface # Ardupilot/PX4 high-level actions (Takeoff, Orbit, Offboard, Land) │ │ ├── drone_traffic_client # Subscriber of topic `/dtc_commands` enforcing high-level actions from the ground │ │ ├── imu_publisher # Multiplexer between PX4/DDS and ArduPilot/MAVROS sensor topics │ │ ├── mission # Orchestrator of the actions in `autopilot_interface` │ │ ├── offboard_control # Low-level references for the Offboard action in `autopilot_interface` │ │ ├── state_sharing # Publisher of the `/state_sharing_drone_N` topic broadcasted by Zenoh │ │ └── yolo_py # GStreamer video acquisition and publisher of YOLO bounding boxes │ │ │ └── aircraft.yml.erb # Aircraft docker tmux entrypoint │ ├── ground │ ├── ground_ws │ │ └── src │ │ ├── drone_traffic_controller # Publisher of topic `/dtc_commands` broadcasted by Zenoh │ │ └── ground_system # Publisher of topic `/tracks` broadcasted by Zenoh │ │ │ └── ground.yml.erb # Ground docker tmux entrypoint │ ├── simulation │ ├── simulation_resources │ │ ├── aircraft_models │ │ │ ├── alti_transition_quad # ArduPilot VTOL model │ │ │ ├── iris_with_ardupilot # ArduPilot quad model │ │ │ ├── sensor_camera # Camera model │ │ │ ├── sensor_gimbal # 3D gimbal used with sensor_camera │ │ │ ├── sensor_lidar # LiDAR model │ │ │ ├── standard_vtol # PX4 VTOL model │ │ │ ├── x500 # PX4 quad model │ │ │ └── sensor_config.yaml # Intrinsics and extrinsics for all sensor and vehicle models │ │ └── simulation_worlds │ │ ├── apple_orchard.sdf │ │ ├── impalpable_greyness.sdf │ │ ├── shibuya_crossing.sdf │ │ ├── swiss_town.sdf │ │ └── waterworld.sdf │ │ │ └── simulation.yml.erb # Simulation docker tmux entrypoint │ └── tools_and_docs ├── docker │ ├── aircraft.dockerfile # Docker image for aircraft simulation and deployment │ ├── ground.dockerfile # Docker image for ground system simulation and deployment │ └── simulation.dockerfile # Docker image for SITL and HITL simulation │ ├── deploy_build.sh # Build `aircraft.dockerfile` for arm64/Orin ├── deploy_run.sh # Start the aircraft docker on arm64/Orin or the ground docker on amd64 (deploy or HITL) │ ├── gym_run.py # Examples for the Gymnasium aas-gym package │ ├── sim_build.sh # Build all dockerfiles for amd64/simulation └── sim_run.sh # Start the simulation (SITL or HITL)
Dependencies management (click to expand)
- Host OS: Ubuntu 22.04/24.04/26.04 (LTS, ESM 4/2036)
- Jetpack: 6.2.1 (rev. 1) [L4T 36.4.4, Ubuntu 22-based]
- TODO: upgrade to JP 7.2 [L4T 39.2, Ubuntu 24-based]
-
nvidia-driver-580 - Docker Engine v29
- NVIDIA Container Toolkit 1.19
-
amd64base image:cuda:12.9.2-cudnn-runtime-ubuntu22.04 -
arm64/Jetson base image:l4t-jetpack:r36.4.0 - DeepStream 7.1
- ROS2 Humble (LTS, EOL 5/2027)
- Gazebo Sim Harmonic (LTS, EOL 9/2028)
- PX4 1.17.0
- ArduPilot 4.6.3
- Ultralytics 8.4/YOLO26
- ONNX Runtime 1.23.2
Transitive constraints (as of May 2026):
- Jetson Orin supports JetPack only up to version 6
- JetPack 6/Orin latest supported DeepStream version is 7.1 (DS 8.0, 9.0 are on JetPack 7/Thor)
- JetPack 6 is based on L4T 36 (Ubuntu 22)
- Ubuntu 22's system Python is version 3.10
- The last available ONNX Runtime GPU wheel for Python 3.10 is version 1.23.2 (ORT 1.24+ is available on Python 3.11+)
- ONNX Runtime GPU 1.23.2 only supports CUDA 12 (CUDA 13 added in ORT 1.24.1)
- The latest CUDA 12 on the NVIDIA NGC Catalog is 12.9.2 (e.g.,
cuda:12.9.2-cudnn-runtime-ubuntu22.04)
- The latest CUDA 12 on the NVIDIA NGC Catalog is 12.9.2 (e.g.,
- ONNX Runtime GPU 1.23.2 only supports CUDA 12 (CUDA 13 added in ORT 1.24.1)
- The last available ONNX Runtime GPU wheel for Python 3.10 is version 1.23.2 (ORT 1.24+ is available on Python 3.11+)
- Ubuntu 22's GStreamer
aptpackage is version 1.20- GStreamer 1.20's
nvh264enc presets are no longer supported beyondnvidia-driver-580;nvidia-driver-595requires GStreamer 1.24, which is the default on Ubuntu 24 (see PR #84)
- GStreamer 1.20's
- Ubuntu 22's system Python is version 3.10
External repositories:
PX4/PX4-Autopilottag/branch:v1.17.0PX4/px4_msgstag/branch:release/1.17PX4/flight_reviewtag/branch:mainArduPilot/ardupilottag/branch:Copter-4.6.3ArduPilot/ardupilot_gazebotag/branch:mainsrmainwaring/asv_wave_simtag/branch:mastermavlink/c_library_v2tag/branch:mastermavlink-router/mavlink-routertag/branch:mastereProsima/Micro-XRCE-DDS-Agenttag/branch:masterPRBonn/kiss-icptag/branch:mainmicrosoft/onnxruntimetag/branch:v1.23.2Livox-SDK/Livox-SDK2tag/branch:masterLivox-SDK/livox_ros_driver2tag/branch:master
You've done a man's job, sir. I guess you're through, huh?































