Perception This page describes the interface of the Perception component. See the Perception component design for the concepts and the reference implementation for the data flow inside the component.
Perception takes sensor data, the maps, the kinematic state of the vehicle, the route, and external traffic signals. It has four parts: obstacle segmentation, occupancy grid map, object recognition, and traffic light recognition. It sends the obstacle point cloud, the occupancy grid, the objects, and the traffic signals to Planning and Control. The labels also name the message types. Sensing LiDAR PointCloud2 Cameras Image Radars DetectedObjects Map Lanelet2 LaneletMapBin Point cloud map Localization Kinematic state Odometry Planning Route LaneletRoute V2X Traffic signals TrafficLightGroupArray Perception Obstacle segmentation Occupancy grid map Object recognition Detection Tracking Prediction Traffic light recognition Cameras and V2X Obstacle point cloud PointCloud2 Occupancy grid OccupancyGrid Objects PredictedObjects TrackedObjects Traffic signals TrafficLightGroupArray Planning and Control The inputs, the main parts, and the outputs of the Perception component. From Sensing Name Topic Type Description Point cloud /sensing/lidar/concatenated/pointcloudsensor_msgs/msg/PointCloud2 The concatenated point cloud of all LiDARs, after preprocessing. Camera image /sensing/camera/camera*/image_rect_colorsensor_msgs/msg/Image Camera images after lens distortion correction. The camera-based object detectors use them. Camera image /sensing/camera/camera*/image_rawsensor_msgs/msg/Image Camera images without lens distortion correction. Traffic light recognition uses them. Radar objects /sensing/radar/detected_objectsautoware_perception_msgs/msg/DetectedObjects The objects of all radars in one topic. The sensor kit merges the objects of each radar into it. See Sensing . Only the object recognition modes with radar use them.
From Map Use the service for the point cloud map when you can. With the service, the size of the map file does not limit the processing.
From Localization Name Topic Type Description Vehicle odometry /localization/kinematic_statenav_msgs/msg/Odometry The pose and the velocity of the vehicle.
From Planning Name Topic Type Description Route /planning/mission_planning/routeautoware_planning_msgs/msg/LaneletRoute Traffic light recognition uses the route to select the traffic lights that matter to the vehicle.
From outside Autoware Name Topic Type Description External traffic signals /perception/traffic_light_recognition/external/traffic_signalsautoware_perception_msgs/msg/TrafficLightGroupArray Traffic signals from an external system, for example V2X. The traffic light arbiter merges them with the recognized signals.
Outputs See the Perception component design for the meaning of each output.
Name Topic Type Description Dynamic objects /perception/object_recognition/objectsautoware_perception_msgs/msg/PredictedObjects Objects that were not in the map when it was made, with their class, shape, and predicted paths. Tracked objects /perception/object_recognition/tracking/objectsautoware_perception_msgs/msg/TrackedObjects The same objects before prediction. The diffusion planner uses them. Obstacles /perception/obstacle_segmentation/pointcloudsensor_msgs/msg/PointCloud2 The points that are not ground. They include dynamic objects and static obstacles that the vehicle must avoid or stop in front of. Occupancy grid map /perception/occupancy_grid_map/mapnav_msgs/msg/OccupancyGrid The probability that each cell holds an obstacle. Cells that the sensors cannot see are unknown, which shows the blind spots. Traffic signals /perception/traffic_light_recognition/traffic_signalsautoware_perception_msgs/msg/TrafficLightGroupArray The state of each traffic light group, for example the color (green, amber, red) and the arrows (left, right, straight).
Predicted objects message autoware_perception_msgs/msg/PredictedObjects :
std_msgs/Header header PredictedObject [] objectsunique_identifier_msgs/UUID object_id float32 existence_probability ObjectClassification [] classificationuint8 label: UNKNOWN, CAR, TRUCK, BUS, TRAILER, MOTORCYCLE, BICYCLE, PEDESTRIAN, ANIMAL, HAZARD, OVER_DRIVABLE, or UNDER_DRIVABLE float32 probability PredictedObjectKinematics kinematicsgeometry_msgs/PoseWithCovariance initial_pose_with_covariance geometry_msgs/TwistWithCovariance initial_twist_with_covariance geometry_msgs/AccelWithCovariance initial_acceleration_with_covariance PredictedPath [] predicted_pathsgeometry_msgs/Pose[] path builtin_interfaces/Duration time_step float32 confidence Shape shapeuint8 type: BOUNDING_BOX, CYLINDER, or POLYGON geometry_msgs/Polygon footprint geometry_msgs/Vector3 dimensions Traffic signals message autoware_perception_msgs/msg/TrafficLightGroupArray :
builtin_interfaces/Time stamp TrafficLightGroup [] traffic_light_groupsint64 traffic_light_group_id TrafficLightElement [] elementsuint8 color: UNKNOWN, RED, AMBER, GREEN, or WHITE uint8 shape: UNKNOWN, CIRCLE, LEFT_ARROW, RIGHT_ARROW, UP_ARROW, UP_LEFT_ARROW, UP_RIGHT_ARROW, DOWN_ARROW, DOWN_LEFT_ARROW, DOWN_RIGHT_ARROW, or CROSS uint8 status: UNKNOWN, SOLID_OFF, SOLID_ON, or FLASHING float32 confidence PredictedTrafficLightState [] predictions