Sensing component design#

Overview#

The sensing component is a collection of modules that apply primitive preprocessing to the raw sensor data.

This component also defines the input formats of the sensor data.

Role#

Abstraction of data formats to enable usage of sensors from various vendors
Perform common/primitive sensor data processing required by each component

High-level architecture#

This diagram shows the high-level architecture of the sensing component.

Sensing preprocesses the data of each sensor type and sends it to localization and perception.

The dashed line from the camera to localization is optional. Only the camera-based localizers (YabLoc and the AR tag localizer) read the camera image.

A sensor kit launch package (for example sample_sensor_kit_launch) launches the drivers and the preprocessing of one sensor set. autoware_launch selects the package with the sensor_model launch argument and includes its launch/sensing.launch.xml in the /sensing namespace. The sensor kits use two shared launch packages:

autoware_sensor_driver_launch: launch files for the LiDAR drivers of Nebula
autoware_sensing_launch: launch files for the preprocessing, for example the point cloud preprocessing, the IMU corrector, and the radar track filters

Inputs#

Input types#

Sensor data	Message type
Point cloud (LiDARs, depth cameras, etc.)	sensor_msgs/msg/PointCloud2.msg
Image (RGB, monochrome, depth, etc. cameras)	sensor_msgs/msg/Image.msg
Compressed image	sensor_msgs/msg/CompressedImage.msg
Radar scan	radar_msgs/msg/RadarScan.msg
Radar tracks	radar_msgs/msg/RadarTracks.msg
Radar objects	autoware_sensing_msgs/msg/RadarObjects.msg
GNSS-INS position	sensor_msgs/msg/NavSatFix.msg
GNSS-INS orientation	autoware_sensing_msgs/GnssInsOrientationStamped.msg
GNSS-INS velocity	geometry_msgs/msg/TwistWithCovarianceStamped.msg
GNSS-INS acceleration	geometry_msgs/msg/AccelWithCovarianceStamped.msg
IMU	sensor_msgs/msg/Imu.msg
Ultrasonics	sensor_msgs/msg/Range.msg
Vehicle velocity	autoware_vehicle_msgs/msg/VelocityReport.msg

Sensing modules#

This table lists the modules that the sensor kits of autoware_launch use, and the topics that they publish.

Data	Package	Processing	Main output topic
Vehicle velocity	`autoware_vehicle_velocity_converter` (Core)	Converts `VelocityReport` to `TwistWithCovarianceStamped`	`/sensing/vehicle_velocity_converter/twist_with_covariance`
GNSS/INS	`autoware_gnss_poser` (Core)	Converts the position to a pose of `base_link` in the map frame	`/sensing/gnss/pose_with_covariance`
IMU	`autoware_imu_corrector` (Universe)	Corrects the gyro offset, scale, and standard deviation. Estimates the gyro bias.	`/sensing/imu/imu_data`
LiDAR	`autoware_pointcloud_preprocessor` or `autoware_cuda_pointcloud_preprocessor` (Universe)	Crops, corrects the motion distortion, removes outliers, and concatenates the point clouds	`/sensing/lidar/concatenated/pointcloud`
Camera	None in the sensing component	The perception and localization modules read the image topics of the camera driver	Driver topics, for example `/sensing/camera/traffic_light/image_raw`
Radar	`autoware_radar_*` packages (Universe)	Filters the radar noise and converts the radar data to Autoware messages	`/sensing/radar/detected_objects`

/sensing/radar/detected_objects is the default radar input of the perception launch files. None of the sensor kits in autoware_launch has a radar.

Design by data types#

GNSS/INS data

The GNSS poser converts the GNSS fix and the INS orientation to a pose in the map frame.

Image

The camera drivers publish the images. The perception modules decompress them when necessary.

Point cloud

The point cloud preprocessing pipeline and the point fields that the LiDAR drivers must publish.

Radar pointcloud data

The noise filter and the static/dynamic filter for RadarScan, and the conversion to PointCloud2.

Radar objects data

The noise filter for RadarTracks, the conversion to DetectedObjects, and the object mergers.

Ultrasonics data

The message type for ultrasonic sensors. Autoware has no ultrasonic processing today.