Image pre-processing design#
Overview#
A camera driver publishes the camera images and the camera information. The sensing component of Autoware has no image preprocessing node in the default launch files. The perception and localization modules read the image topics of the camera driver directly.
The camera driver is not part of Autoware. The driver, or another node outside Autoware, must publish the topics that the next sections describe. In the AWSIM and CARLA sensor kits, the simulator publishes these topics.
Message types#
| Data | Message type | Description |
|---|---|---|
| Image | sensor_msgs/msg/Image | An uncompressed image, for example image_raw or image_rect_color |
| Compressed image | sensor_msgs/msg/CompressedImage | A compressed image, for example image_raw/compressed |
| Camera info | sensor_msgs/msg/CameraInfo | The intrinsic parameters and the distortion of the camera |
Topics that Autoware reads#
All the camera topics are in the /sensing/camera/<camera name>/ namespace.
| User | Topics | Notes |
|---|---|---|
| Camera-based object detection | /sensing/camera/camera<N>/image_rect_color and camera_info | The image must be rectified (the lens distortion is removed). Autoware does not rectify the image. |
| Traffic light recognition | /sensing/camera/<camera name>/image_raw and camera_info | The launch files use camera6 and camera7 as the representative cameras. |
| YabLoc and AR tag localizers (optional) | /sensing/camera/traffic_light/image_raw | Only when pose_source includes yabloc or artag |
The camera.launch.xml file of sample_sensor_kit_launch shows an example. It relays the compressed image and the camera information of the traffic light camera to /sensing/camera/traffic_light/image_raw/compressed and /sensing/camera/traffic_light/camera_info. The sensing.launch.xml file of this sensor kit does not include camera.launch.xml by default.
Decompression#
Some camera drivers publish only compressed images. The traffic light recognition launch files load autoware_image_transport_decompressor when enable_image_decompressor is true (default). This node reads <image topic>/compressed and publishes the decompressed image on <image topic>.
The node supports 8-bit RGB and BGR cameras, with the encoding parameter set to rgb8 or bgr8. For other encodings, the published image does not match the camera data. The package README describes each case.
Other image packages#
The sensing directory of Autoware Universe has two more image packages. The launch files of autoware_launch do not launch them.
autoware_image_diagnosticsdivides each raw image into blocks. It finds blocks with shadow clipping, highlight clipping, blockage, or low visibility, and publishes the image state on/diagnostics.autoware_calibration_status_classifierprojects the LiDAR points onto the undistorted camera image. A neural network then classifies the LiDAR-camera calibration as correct or not.