Skip to content

Architecture 2.0#

Architecture 2.0 is the architecture of Autoware 2.0. It adds learned planning next to the rule-based planning stack. The design is the generator-selector architecture: a generator makes candidate trajectories, and a selector checks the candidates and picks one.

The rule-based stack (sensing, map, localization, perception, planning, control, and vehicle) is still the default stack in Autoware 2.0.0. Architecture 2.0 does not replace it. The rule-based planning stack becomes one generator among others.

  • Components

    The design of each component of the rule-based stack.

  • Interfaces

    The messages between the components, and the AD API for external systems.

  • Node diagram

    All the nodes and topics of Autoware in one diagram.

  • Roadmap

    The steps from the robotics stack to learned driving.

The generator-selector architecture#

Sensing sends sensor data to the generators. The generators send candidate trajectories to the selector. The selector checks the candidates with a safety gate, ranks them, and sends the selected trajectory to the controller, which drives the vehicle.SensingLiDAR, camera, radarVehicleControllerFollows the trajectorySensor dataGeneratorsAny module that outputs trajectories. They can run in parallel.Robotics approachRule-based planning stackModular E2ELearned perception, planningMonolithic E2EOne network, sensors to pathWorld model E2ELearns how scenes evolveCandidate trajectoriesSelectorChecks every candidate, then picks oneSafety gateRankingSelected trajectory
Generators make candidate trajectories. The selector checks them and picks one for the controller.

A traditional autonomous driving stack is a fixed pipeline:

  • Sensing
  • Perception
  • Localization
  • Planning
  • Trajectory

This pipeline works well for rule-based planners. Newer approaches, for example end-to-end (E2E) models and diffusion models, do not fit this structure. They can skip or replace parts of the pipeline, so it is difficult to integrate them. To support both classical and newer approaches, Architecture 2.0 abstracts away the front half of the pipeline.

  • Generators make trajectories

    A generator is any module that outputs candidate trajectories: a rule-based or optimization planner that uses perception and maps, an E2E model that uses raw sensor data, or a learned or sampling-based planner. A generator can use the sensing, perception, localization, and control of Autoware, or it can bypass them. Many generators can run in parallel.

  • The selector makes the final choice

    The selector receives the candidate trajectories. First, it does safety checks on them, for example traffic rules and the drivable area. Then it ranks the safe candidates and selects the best one for the context or the driving policy.

What this architecture makes possible

  • Robotics planners and E2E planners work in the same stack.
  • Explicit checks make the use of black-box models safer.
  • You can try a new planning method without changes to the rest of the stack.
  • The selector can compare many trajectory proposals before it decides.

The Generator-Selector framework page gives the full design.

What Autoware 2.0.0 does today#

In Autoware 2.0.0, you choose the planner when you launch Autoware. The planning_setting launch argument has two values:

  • rule_based (default): the rule-based scenario planner makes the trajectory, as in Autoware 1.x.
  • diffusion_planner: the Diffusion Planner, a learned planner, makes the trajectory.

Only one generator runs at a time. The selector packages are in autoware_universe, but Autoware 2.0.0 does not launch them yet. The launch file has a placeholder where the selector goes. Until then, autoware_trajectory_modifier passes the first candidate trajectory to the planning validator.

Data flow#

The planning data flow in Autoware 2.0.0. With planning_setting set to diffusion_planner, the Diffusion Planner sends candidate trajectories to the trajectory modifier, which sends the first candidate to the planning validator. With rule_based, scenario planning sends its trajectory to the planning validator. The validator publishes /planning/trajectory for control. The trajectory selector is not launched.planning_setting:=diffusion_plannerautoware_diffusion_plannerGenerator: a learned plannerCandidateTrajectoriesautoware_trajectory_modifierModifier and optimizer pluginsTrajectory selectorNot launched in 2.0.0planning_setting:=rule_basedThe default settingScenario planningRule-based planning stackautoware_planning_validatorChecks the trajectoryTrajectory (first candidate)TrajectoryControl/planning/trajectory
The planning_setting launch argument selects the source of the trajectory. Both sources go through the planning validator.

With planning_setting:=diffusion_planner, the data flows as follows:

  1. autoware_diffusion_planner publishes an autoware_internal_planning_msgs/msg/CandidateTrajectories message on /planning/generator/diffusion_planner/candidate_trajectories.
  2. autoware_trajectory_modifier applies its modifier and optimizer plugins to each candidate. It publishes the first candidate on /planning/trajectory_generator/trajectory. It also publishes all the processed candidates on /planning/generator/candidate_trajectories.
  3. autoware_planning_validator reads this trajectory and publishes /planning/trajectory for the control stack.

With planning_setting:=rule_based, the planning validator reads /planning/scenario_planning/velocity_smoother/trajectory instead.

Packages#

RolePackageStatus in 2.0.0
Generatorautoware_diffusion_plannerRuns with planning_setting:=diffusion_planner
Post-processingautoware_trajectory_modifierRuns with planning_setting:=diffusion_planner
Backup generatorautoware_minimum_rule_based_plannerNot launched. It follows the lanelet centerline and publishes CandidateTrajectories.
Selector nodeautoware_trajectory_selectorNot launched. It concatenates, validates, and ranks the candidates, and publishes one Trajectory. It uses the four libraries below.
Concatenationautoware_trajectory_concatenatorNot launched. It is a library inside autoware_trajectory_selector. It merges the candidates of all generators.
Safety gateautoware_trajectory_validatorNot launched. It is a library inside autoware_trajectory_selector. Its plugins remove unsafe candidates and candidates that break rules.
Rankingautoware_trajectory_rankerNot launched. It is a library inside autoware_trajectory_selector. It scores each candidate.
Adapterautoware_trajectory_adapterNot launched. It is a library inside autoware_trajectory_selector. It converts the best candidate to a Trajectory.
Messagesautoware_internal_planning_msgsCandidateTrajectories, CandidateTrajectory, GeneratorInfo, ScoredCandidateTrajectories, and ScoredCandidateTrajectory

The launch files are in autoware_planning_launch in autoware_launch.

Run the Diffusion Planner in the planning simulator#

The Diffusion Planner uses TensorRT, so you need an NVIDIA GPU.

  1. Follow the planning simulation preparation steps. They download the sample map and make sure that the AI models are on your computer.

  2. Make sure that the model folder exists:

    ls ~/autoware_data/ml_models/diffusion_planner/v5.0/

  3. Launch the planning simulator with the Diffusion Planner:

    source ~/autoware/install/setup.bash
ros2 launch autoware_launch planning_simulator.launch.xml \
  map_path:=$HOME/autoware_data/maps/sample-map-planning \
  vehicle_model:=sample_vehicle \
  sensor_model:=sample_sensor_kit \
  planning_setting:=diffusion_planner

  4. Set the initial pose and the goal pose in RViz, as in the planning simulation demo.

At the first launch, the Diffusion Planner builds a TensorRT engine from the ONNX model. As a result, the first start takes longer than the next starts.

AI models#

The Diffusion Planner and the other AI models of Autoware are on Hugging Face. Each model has version tags. The artifacts Ansible role downloads each model at a fixed version with the hf CLI into ~/autoware_data/ml_models/. For example, the role downloads the Diffusion Planner versions v3.0, v3.1, v4.0, and v5.0, and Autoware 2.0.0 uses v5.0.

To download the models, run this command from the autoware directory:

ansible-playbook autoware.dev_env.install_dev_env --tags artifacts --ask-become-pass

Other learned driving packages#

  • autoware_tensorrt_vad is in the e2e/ folder of Autoware Universe. It runs the VAD model, one network that maps six camera images to trajectories. The model is trained on CARLA simulation data. The package has its own launch file, and autoware_launch 0.53.0 does not include it.
  • Vision Pilot, an open-source end-to-end Level 2 ADAS system, is in the Autoware Index. It has its own repository, and the autoware.repos file does not include it.

The rule-based stack#

The rule-based stack has seven components. Their design pages describe the default stack of Autoware 2.0.0.

  • Sensing

    Preprocesses the raw sensor data and defines its input formats.

  • Map

    Gives the vector map and the point cloud map to the rest of the stack.

  • Localization

    Estimates the pose, velocity, and acceleration of the vehicle.

  • Perception

    Recognizes the objects, obstacles, and traffic lights around the vehicle.

  • Planning

    Makes the target trajectory (path and speed) of the vehicle.

  • Control

    Makes the vehicle follow the trajectory.

  • Vehicle

    Converts Autoware commands to the format of the vehicle, and the vehicle status to Autoware messages.

Roadmap#

The roadmap describes the planned steps from the robotics stack to learned driving. Autoware 2.0.0 contains the first step: the Diffusion Planner is a learned planner that you can select at launch.