Skip to content

Data, MLOps, and training#

This page describes how the AI models of Autoware are trained, versioned, and distributed today. It describes only work that exists, with links to the source. Machine learning operations (MLOps) are the tools and processes that train, track, and deliver models.

The Autoware 2.0.0 release does not contain training code. Each model has its own training repository, and the release downloads the trained models.

  • Data
  • Training
  • Versioned model on Hugging Face
  • artifacts role
  • Autoware node

Model distribution#

The Autoware Foundation hosts the AI models of Autoware on Hugging Face. Each model is a Hugging Face repository, and each model version is a tag on that repository.

The artifacts Ansible role of Autoware 2.0.0 downloads each model at a fixed tag with the hf CLI. The default destination is ~/autoware_data/ml_models/. The role downloads the models of these packages and more:

  • Perception, for example lidar_centerpoint, lidar_transfusion, bevfusion, tensorrt_yolox, the traffic light classifiers, and simpl_prediction.
  • Planning: diffusion_planner (tags v3.0, v3.1, v4.0, and v5.0).
  • End-to-end driving: tensorrt_vad (tag v0.1).

A launch file selects one version through a parameter path. For example, the Diffusion Planner of Autoware 2.0.0 reads $(var data_path)/diffusion_planner/v5.0.

The Hugging Face organization also holds models that are not in the release, for example auto_e2e, meteor, oneplanner, and latent_drive. It also holds datasets, for example meteor-demo-scenes, CARLA maps, and the autoware-ai-archive recordings.

Training of the models in Autoware Universe#

PackageModel and training
autoware_diffusion_plannerDiffusion Planner, from the paper "Diffusion-Based Planning for Autonomous Driving with Flexible Guidance". The tier4/Diffusion-Planner fork trains the model.
autoware_tensorrt_vadVAD (Vectorized Autonomous Driving). The model is trained on the Bench2Drive dataset, which is CARLA simulation data. NVIDIA DL4AGX optimizes it for TensorRT.

The README of each package gives the model versions and the links to the original papers.

Training projects of the Autoware Foundation#

AutoE2E#

AutoE2E is an open-source, camera-first E2E driving model. The repository holds the full training stack:

  • The Model folder holds the model, the data processing, the training, the distributed training, and the evaluation code.
  • The Platform folder holds an MLOps platform on Amazon EKS. It uses MLflow to track experiments, Flyte to run the pipelines, and Kueue to queue the GPU jobs. The datasets are WebDataset shards in Amazon S3.
  • The Tools folder holds the DataModelConsole, a dashboard that replays camera scenes with the ground-truth and the predicted trajectories.

The released checkpoints are trained on nuPlan, and one checkpoint is fine-tuned on KITScenes-Multimodal. The checkpoints are on Hugging Face.

METEOR#

METEOR is a reference E2E model for surround-view driving from TIER IV. Its repository holds the source, the training recipe, an autolabel factory, and the runtimes. The autolabel factory makes all the training labels from raw recordings, without human labels. The weights are on Hugging Face. The announcement is the discussion METEOR: an open reference E2E model for surround-view driving.

Data tools#